-- Logs begin at Tue 2022-07-19 19:49:03 UTC, end at Tue 2022-07-19 21:17:51 UTC. -- Jul 19 20:33:33.623495 dv-cinder-799871-37-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 19 20:33:33.633057 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 19 20:33:33.635091 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 19 20:33:33 2022] *** Jul 19 20:33:33.635091 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 19 20:33:33.635091 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 19 20:33:33.635091 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: nodename: dv-cinder-799871-37-iscsi Jul 19 20:33:33.635091 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: machine: x86_64 Jul 19 20:33:33.635091 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: clock source: unix Jul 19 20:33:33.635091 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: pcre jit disabled Jul 19 20:33:33.635091 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: detected number of CPU cores: 4 Jul 19 20:33:33.635091 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: current working directory: / Jul 19 20:33:33.635091 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: detected binary path: /usr/bin/uwsgi-core Jul 19 20:33:33.635611 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: your processes number limit is 47680 Jul 19 20:33:33.635611 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: your memory page size is 4096 bytes Jul 19 20:33:33.635611 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: detected max file descriptor number: 2048 Jul 19 20:33:33.635611 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: lock engine: pthread robust mutexes Jul 19 20:33:33.635611 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: thunder lock: enabled Jul 19 20:33:33.635611 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 19 20:33:33.635611 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 19 20:33:33.679078 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: Python main interpreter initialized at 0x564c56f809c0 Jul 19 20:33:33.679078 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: python threads support enabled Jul 19 20:33:33.679078 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: your server socket listen backlog is limited to 100 connections Jul 19 20:33:33.679078 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: your mercy for graceful operations on workers is 90 seconds Jul 19 20:33:33.679436 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: mapped 403077 bytes (393 KB) for 2 cores Jul 19 20:33:33.679511 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: *** Operational MODE: preforking *** Jul 19 20:33:33.679781 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: *** uWSGI is running in multiple interpreter mode *** Jul 19 20:33:33.679781 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: spawned uWSGI master process (pid: 92300) Jul 19 20:33:33.679988 dv-cinder-799871-37-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 19 20:33:33.681084 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: spawned uWSGI worker 1 (pid: 92310, cores: 1) Jul 19 20:33:33.681084 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: spawned uWSGI worker 2 (pid: 92311, cores: 1) Jul 19 20:33:33.681084 dv-cinder-799871-37-iscsi devstack@keystone.service[92300]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 19 20:33:39.216488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 20:33:39.216488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: from cryptography.utils import int_from_bytes Jul 19 20:33:39.217332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 20:33:39.217332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: from cryptography.utils import int_from_bytes Jul 19 20:33:39.217523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 20:33:39.217523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: from cryptography.utils import int_from_bytes Jul 19 20:33:39.217943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 20:33:39.217943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: from cryptography.utils import int_from_bytes Jul 19 20:33:39.266920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 20:33:39.267115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 20:33:39.267206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 20:33:39.267299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 20:33:39.267374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] command line args: [] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 20:33:39.267453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] command line args: [] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 20:33:39.267565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 20:33:39.267723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 20:33:39.267812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ================================================================================ {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 20:33:39.267929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ================================================================================ {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 20:33:39.268017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] admin_token = **** {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.268238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] admin_token = **** {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.268326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] config_dir = [] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.268470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.268557 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] config_dir = [] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.268665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] config_source = [] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.268753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.268862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] debug = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.268949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] config_source = [] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.269141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.269524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] debug = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.269524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.269756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.269874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.269991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.270109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.270232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.270347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.270461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.270585 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.270703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] list_limit = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.270824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] list_limit = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.270824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] log_config_append = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.271054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] log_config_append = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.271166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.271277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.271398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] log_dir = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.271517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] log_dir = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.271638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] log_file = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.271760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] log_file = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.271862 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.271974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.272079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.272185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.272317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.272317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.272520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.272644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.272762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.272889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.273020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.273148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.273263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.275194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.275393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.275599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.275738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.275858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.275858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.276098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.276226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.276348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.276464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.276603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.276722 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.276852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.276968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.277091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.277091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.277413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.277413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] publish_errors = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.277696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.277823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.277823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.277823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.278149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.278149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] publish_errors = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.278372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.278372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.278608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.278738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] standard_threads = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.278896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.279045 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.279171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.279301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.279425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.279620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.279758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.279879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] use_journal = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.279879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] use_json = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.280119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] standard_threads = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.280244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] use_stderr = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.280244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] use_syslog = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.280491 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.280614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.280751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.280889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.281556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.281685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.281809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] use_journal = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.281809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] use_json = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.282057 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.282180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] use_stderr = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.282269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.282342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] use_syslog = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.282419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.282496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:33:39.282573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.282645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.282726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.282726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.282854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.282924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.282999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.283933 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284094 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284307 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.284961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285599 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285863 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.285942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.286019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.286090 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.286159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.286228 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.286297 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.286426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.286503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.286618 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.286701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.286818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.286944 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.287026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.287193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.287276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.287428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.287510 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.287668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.287758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.287915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.288040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.288149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.288331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.288421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.288599 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.288678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.288828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.288905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.289066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.289223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.289297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.289429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.289561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.289707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.289823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.289998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.290106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.290240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.290346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.290478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.290586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.290737 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.290853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.290971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.291124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.291203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.291372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.291464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.291610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.291692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.291848 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.291926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.292090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.292181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.292319 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.292399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.292567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.292684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.292793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.292877 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.293027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.293107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.293261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.293384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.293538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.293630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.293773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.293915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.293991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.294108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.294234 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.294324 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.294472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.294559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.294711 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.294791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.294954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.295037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.295196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.295280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.295431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.295512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.295666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.295745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.295902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.295990 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.296139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.296219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.296373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.296461 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.297234 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.297335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.297478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.297602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.297718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.297886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.297968 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.298158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.298239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.298425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.298507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.298677 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.298764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.298931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.299013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.299177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.299262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.299416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.299504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.299657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.299738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.299896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.299976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.300148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.300236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.300390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.300470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.300630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.300712 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.300873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.300964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.301113 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.301192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.301345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.301474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.301604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.301770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.301882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.301971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.302161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.302275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.302422 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.302571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.302723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.302825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.302906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.303017 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.303106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.303271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.303351 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.303515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.303601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.303777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.303858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.304031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.304111 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.304266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.304348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.304498 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] auth.external = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.304578 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.304728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.304807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.304968 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.305050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.305203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.305282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.305473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.305573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] auth.password = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.305716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] auth.token = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.305797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.305959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.306039 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.306197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.306289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.306444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.306535 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.306677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.306810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.306925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.307107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.307188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.307353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] auth.external = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.307447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.307589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.307670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] credential.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.307853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.307933 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.308102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.308185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.308333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] auth.password = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.308413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.308568 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] auth.token = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.308649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.308810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.308891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.309045 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.309126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.309298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.309415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.309547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.309631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.309796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.309878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.310046 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.310126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.310281 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.310363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.310520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] credential.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.310601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.310771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.310851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.311025 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.311108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.311277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.311358 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.311511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] federation.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.311592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.311749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.311832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.311984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.312066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.312230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.312318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.312460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.312542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.312704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.312785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.312949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.313030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.313203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.313286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.313465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.313548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.313720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.313801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.313956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.314039 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.314204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.314286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.314438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.314521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] federation.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.314683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.314766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.314938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.315019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.315172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.315253 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.315407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.315490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.315686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.315775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.315926 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.316009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.316190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.316275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.316433 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.316514 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.316693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.316782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.316924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.317006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.317164 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.317251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.317422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.317569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.317676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.317837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.317919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.318155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.318243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.318397 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.318481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.318701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.318782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.318956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.319082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.319222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.319304 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.319462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.319549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.319705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.319786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.319944 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.320066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.320176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.320258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.320405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.320484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.320643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.320725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.320883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.320962 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.321122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.321201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.321398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.321483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.321626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.321714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.321873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.321953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.322121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.322203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.322358 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.322438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.322593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.322672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.322838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.322919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.323067 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.323146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.323305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.323383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.323533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.323612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.323771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.324118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.324274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.324353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.324515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.324596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.324754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.324833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.324991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.325070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.325231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.325311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.325506 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.325599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.325738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.325817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.325969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.326049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.326213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.326291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.326448 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.326527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.326680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.326759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.326918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.326995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.327150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.327227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.327380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.327461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.327618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.327697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.327891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.327976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.328128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.328208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.328373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.328452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.328604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.328682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.328851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.328934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.329112 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.329198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.329314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.329424 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.329575 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.329655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.329809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.329888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.330042 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.330172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.330267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.330346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.330511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.330589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.330754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.330841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.330997 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.331077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.331240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.331318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.331482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.331568 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.331713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.331792 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.331944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.332022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.332183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.332271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.332417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.332497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.332652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.332731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.332882 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.332969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.333134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.333213 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.333401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.333474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.333662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.333783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.333908 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.333998 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.334157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.334240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.334400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.334490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.334635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.334716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.334871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.334952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.335107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.335197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.335345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.335425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.335584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.335664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.335815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.335905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.336052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.336134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.336290 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] resource.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.336371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.336537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.336626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.336776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.336856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.337009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.337090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.337256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.337349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.337527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.337613 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.337817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.337899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.338020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.338117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.338267 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.338350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.338504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.338587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.338742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] role.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.338831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.338984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] resource.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.339116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] role.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.339246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.339403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.339533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.339737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.339867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.339995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.340122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.340249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.340375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.340513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.340640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.340779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.340907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.341034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.341159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.341296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.341468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.341595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.341723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.341852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] role.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.341979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.342116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] role.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.342243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.342371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.342511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.342640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.342776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.342904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.342904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.343142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.343268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.343393 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.344020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.344149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.344276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.344401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.344527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.344664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.344799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.344927 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.345055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.345180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.345307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.345457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.345598 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.345728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.345854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.345980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.346107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.346233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.346233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.346493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.346639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.346764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.346891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.347018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.347154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.347283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.347415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.347543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.347671 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.347797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.347933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.348060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.348196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.348323 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.348449 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] token.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.348579 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.348716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.348846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.348924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.348996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.349068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.349148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.349221 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.349293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.349380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.349502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.349580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.349665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.349740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.349812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.349930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350234 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] token.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.350926 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.351008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.351163 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.351245 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.351425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.351505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.351668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.351749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.connection = **** {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.351918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.352014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.352165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.352250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.352401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.352482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.352640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.352722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.352877 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.352967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.353112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.353191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.353345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.353444 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.353602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.353686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.353845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.353925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.354079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.354160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.354321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.354403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.354555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.354673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.connection = **** {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.354802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.354893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.355036 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.355127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.355280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.355362 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.355509 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.355648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.355731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.355857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.355966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.356048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.356120 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92310) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 20:33:39.356232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.356472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.356721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.357009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.357258 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.357528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.357781 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.358021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.358271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.358505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.358749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.358988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:33:39.359212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92311) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 20:33:39.388677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.388900 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.389119 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.389610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.389852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.390071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.397622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.398552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.426920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.427192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.427613 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.427855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.573314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.574452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.623298 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.623597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.623737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.623903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.689704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.689971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.690117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.690272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.690408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.690539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.690671 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.690814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.690953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.691081 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.691209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.691335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.715759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.716002 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.716157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.716291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.716419 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.716558 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.716686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.716822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.716952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.717078 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.717204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.717329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.717494 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.717494 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.728359 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.728817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.741672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.741906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.742037 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.742171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.742299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.742427 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.742566 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.742695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.742823 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.742955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.743083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.743219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92310) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.743359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.743359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92311) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:33:39.760643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.760872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 20:33:39.761165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.761319 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.761479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 20:33:39.761605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 20:33:39.761739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.761870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.762013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.762156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 20:33:39.762285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.762414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.762543 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.762672 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.762810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.762977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.773538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.773813 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.774079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.774220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.774367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.774501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.774696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.774978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.782572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.783229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.785847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.786131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.786403 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.786569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.786796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.786928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.787079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.787209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.787335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.787513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.787652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.787807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.787934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.788064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.788190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.788345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.788479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.788610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.788735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.788865 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.789115 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.789420 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.816017 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.817036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.817657 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.817938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.818715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.819001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.823276 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.823597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.824092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.824547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.824827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.825110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.831371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.831652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.832478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.832766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.836237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.836521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.836795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.837070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.837339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.837493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.837627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.837757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.837887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.838038 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.838169 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.838299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.838532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.838813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.839082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.839353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.856660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.856937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.858013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.858297 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.862154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.863454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.865788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.866082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.867134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.867436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.871070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.872457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.874679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.876657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.879060 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.879365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.879654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.879935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.880223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.880457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.880614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.880761 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.881059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.881384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.881664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.881954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.899339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.899713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.899987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.900250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.900877 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.901260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.901574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.901843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.908587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.910282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.910424 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.910600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.911998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.912277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.913692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.913971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.914251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.914527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.914799 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.915374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.915660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.915946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.916243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.916565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.928746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.929022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.929299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.929884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.930177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.930469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.930582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.930863 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.931144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.931426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.931713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.932005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.932293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.950046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.951942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.953306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.955290 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.956602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.958275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.958647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.959895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.960180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.960263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.960459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.961920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.962206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.962499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.967841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.970058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.970542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.972162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.973240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.973559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.973935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.974187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.975441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.975730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.976019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.976298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.983337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.983621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.983898 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.986176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.986468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.986752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.992374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92310) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.992649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.992925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.993196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.993499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.993771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.994055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.994324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.994596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.994724 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92311) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:33:39.994853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.994980 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.995116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.995277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.995410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.995551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.995737 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.995872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.996013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92310) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.996145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.996271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.996490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.996771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.997045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.997335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.997630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.997900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:39.998169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92311) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:33:40.033766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x564c56f809c0 pid: 92310 (default app) Jul 19 20:33:40.036373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x564c56f809c0 pid: 92311 (default app) Jul 19 20:33:40.037180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b9515-6f5e-4768-9b36-ad668984a3ce None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:40.037532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b9515-6f5e-4768-9b36-ad668984a3ce None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:40.037789 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b9515-6f5e-4768-9b36-ad668984a3ce None None] PATH_INFO: `/v3` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:40.042127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1/1] 10.222.0.70 () {50 vars in 791 bytes} [Tue Jul 19 20:33:40 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:33:47.803336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a6abc-207f-402f-a04b-71a264ce7597 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:47.803889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a6abc-207f-402f-a04b-71a264ce7597 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:47.803889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a6abc-207f-402f-a04b-71a264ce7597 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:47.808237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1/2] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:33:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:33:47.814474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fecfb9-33a3-4743-aee2-ac8fc1a4e750 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:47.814902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fecfb9-33a3-4743-aee2-ac8fc1a4e750 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:47.815024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fecfb9-33a3-4743-aee2-ac8fc1a4e750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:47.883331 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: INFO dbcounter [None req-51fecfb9-33a3-4743-aee2-ac8fc1a4e750 None None] Registered counter for database keystone Jul 19 20:33:47.904179 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG oslo_db.sqlalchemy.engines [None req-51fecfb9-33a3-4743-aee2-ac8fc1a4e750 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92310) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 20:33:47.910387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writer thread running {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 20:33:48.086792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG passlib.handlers.bcrypt [None req-51fecfb9-33a3-4743-aee2-ac8fc1a4e750 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92310) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 19 20:33:48.087135 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG passlib.handlers.bcrypt [None req-51fecfb9-33a3-4743-aee2-ac8fc1a4e750 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92310) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 19 20:33:48.141806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-51fecfb9-33a3-4743-aee2-ac8fc1a4e750 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:33:48.143402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-51fecfb9-33a3-4743-aee2-ac8fc1a4e750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:33:48.264199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 2/3] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:33:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 19 20:33:48.434082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a6abc-207f-402f-a04b-71a264ce7597 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:48.503788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: INFO dbcounter [None req-690a6abc-207f-402f-a04b-71a264ce7597 None None] Registered counter for database keystone Jul 19 20:33:48.523577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG oslo_db.sqlalchemy.engines [None req-690a6abc-207f-402f-a04b-71a264ce7597 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92311) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 20:33:48.529737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writer thread running {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 20:33:48.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a6abc-207f-402f-a04b-71a264ce7597 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:48.652649 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d778b2-e356-40ab-8638-09097d334a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:48.653311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d778b2-e356-40ab-8638-09097d334a6c admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:48.653619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d778b2-e356-40ab-8638-09097d334a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:48.653892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d778b2-e356-40ab-8638-09097d334a6c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:48.663204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d778b2-e356-40ab-8638-09097d334a6c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:48.717524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: WARNING keystone.common.rbac_enforcer.enforcer [None req-51d778b2-e356-40ab-8638-09097d334a6c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 19 20:33:48.718779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d778b2-e356-40ab-8638-09097d334a6c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:48.730761 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 2/4] 10.222.0.70 () {58 vars in 1324 bytes} [Tue Jul 19 20:33:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 20:33:48.736794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fecfb9-33a3-4743-aee2-ac8fc1a4e750 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:48.751500 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fecfb9-33a3-4743-aee2-ac8fc1a4e750 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:48.756188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22499b16-f819-4bb3-a8ff-f710d77c718b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:48.757311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22499b16-f819-4bb3-a8ff-f710d77c718b admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:48.757635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22499b16-f819-4bb3-a8ff-f710d77c718b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:48.757917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22499b16-f819-4bb3-a8ff-f710d77c718b admin admin] PATH_INFO: `/v3/projects/a9ddf9c6d32c459d8f135c30485e0f10` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:48.762097 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22499b16-f819-4bb3-a8ff-f710d77c718b admin admin] RBAC: Authorizing `identity:get_project(project_id=a9ddf9c6d32c459d8f135c30485e0f10)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:48.813603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: WARNING keystone.common.rbac_enforcer.enforcer [None req-22499b16-f819-4bb3-a8ff-f710d77c718b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 19 20:33:48.813955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22499b16-f819-4bb3-a8ff-f710d77c718b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:48.818763 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 3/5] 10.222.0.70 () {56 vars in 1207 bytes} [Tue Jul 19 20:33:48 2022] GET /identity/v3/projects/a9ddf9c6d32c459d8f135c30485e0f10 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:33:50.872610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9976cd9-efd8-4c62-9ff0-c55c04fd9b9c None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:50.873185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9976cd9-efd8-4c62-9ff0-c55c04fd9b9c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:50.873185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9976cd9-efd8-4c62-9ff0-c55c04fd9b9c None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:50.877967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 3/6] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:33:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:33:50.884015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8b2a8-d1ac-408e-938b-50b5d962912a None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:50.884485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8b2a8-d1ac-408e-938b-50b5d962912a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:50.884577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8b2a8-d1ac-408e-938b-50b5d962912a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:50.903647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-1ac8b2a8-d1ac-408e-938b-50b5d962912a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:33:50.905207 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1ac8b2a8-d1ac-408e-938b-50b5d962912a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:33:50.913306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 4/7] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:33:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 19 20:33:50.936438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9976cd9-efd8-4c62-9ff0-c55c04fd9b9c None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:50.949563 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9976cd9-efd8-4c62-9ff0-c55c04fd9b9c None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:50.954160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeb907-4615-4835-bdc1-42c0491b791b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:50.954802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb907-4615-4835-bdc1-42c0491b791b admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:50.955092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb907-4615-4835-bdc1-42c0491b791b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:50.955377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb907-4615-4835-bdc1-42c0491b791b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:50.963639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeb907-4615-4835-bdc1-42c0491b791b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:50.965430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeb907-4615-4835-bdc1-42c0491b791b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:50.974492 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 4/8] 10.222.0.70 () {58 vars in 1324 bytes} [Tue Jul 19 20:33:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:33:50.980379 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8b2a8-d1ac-408e-938b-50b5d962912a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:50.990265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8b2a8-d1ac-408e-938b-50b5d962912a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:50.994718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ba908-eeae-4bfb-a7f1-e8194e04377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:50.995585 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba908-eeae-4bfb-a7f1-e8194e04377e admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:50.995866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba908-eeae-4bfb-a7f1-e8194e04377e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:50.996143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ba908-eeae-4bfb-a7f1-e8194e04377e admin admin] PATH_INFO: `/v3/users/629381bc156b476a94122bdd0a64b60d` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:50.997591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ba908-eeae-4bfb-a7f1-e8194e04377e admin admin] RBAC: Authorizing `identity:get_user(user_id=629381bc156b476a94122bdd0a64b60d)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:50.999997 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ba908-eeae-4bfb-a7f1-e8194e04377e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:51.001613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 5/9] 10.222.0.70 () {56 vars in 1198 bytes} [Tue Jul 19 20:33:50 2022] GET /identity/v3/users/629381bc156b476a94122bdd0a64b60d => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:33:53.192230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048c99-76bb-4d15-b22e-45b763c35dda None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:53.192781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d171e9-6bde-4132-8a14-2402b89235d5 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:53.193129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048c99-76bb-4d15-b22e-45b763c35dda None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:53.193225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d171e9-6bde-4132-8a14-2402b89235d5 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:53.193225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d171e9-6bde-4132-8a14-2402b89235d5 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:53.193350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048c99-76bb-4d15-b22e-45b763c35dda None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:53.194513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 6/10] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:33:53.197409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 5/11] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:33:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:33:53.200879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48627-1938-4e20-be45-3c89e96c51e2 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:53.201166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48627-1938-4e20-be45-3c89e96c51e2 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:53.201479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48627-1938-4e20-be45-3c89e96c51e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:53.202025 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc075c-9e6b-416e-a9d9-e0597fe1cb89 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:53.202364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc075c-9e6b-416e-a9d9-e0597fe1cb89 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:53.202651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc075c-9e6b-416e-a9d9-e0597fe1cb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:53.219699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-47b48627-1938-4e20-be45-3c89e96c51e2 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:33:53.220920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-47b48627-1938-4e20-be45-3c89e96c51e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:33:53.244758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG passlib.handlers.bcrypt [None req-34bc075c-9e6b-416e-a9d9-e0597fe1cb89 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92311) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 19 20:33:53.245195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG passlib.handlers.bcrypt [None req-34bc075c-9e6b-416e-a9d9-e0597fe1cb89 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92311) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 19 20:33:53.291652 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-34bc075c-9e6b-416e-a9d9-e0597fe1cb89 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:33:53.292639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-34bc075c-9e6b-416e-a9d9-e0597fe1cb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:33:53.312779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 7/12] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:33:53.336271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b48627-1938-4e20-be45-3c89e96c51e2 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:53.361664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:53.412327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 6/13] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:33:53.436131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc075c-9e6b-416e-a9d9-e0597fe1cb89 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:53.453062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b48627-1938-4e20-be45-3c89e96c51e2 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:53.491737 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974160ec-ff31-436a-9b9f-7fc20414b7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:53.492427 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974160ec-ff31-436a-9b9f-7fc20414b7f7 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:53.492709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974160ec-ff31-436a-9b9f-7fc20414b7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:53.492986 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974160ec-ff31-436a-9b9f-7fc20414b7f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:53.548891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc075c-9e6b-416e-a9d9-e0597fe1cb89 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:53.551866 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:53.580812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974160ec-ff31-436a-9b9f-7fc20414b7f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:53.582322 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974160ec-ff31-436a-9b9f-7fc20414b7f7 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:53.589468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef74bb-00f6-4c81-bc2e-7d369691c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:53.590611 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef74bb-00f6-4c81-bc2e-7d369691c159 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:53.590960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef74bb-00f6-4c81-bc2e-7d369691c159 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:53.591305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef74bb-00f6-4c81-bc2e-7d369691c159 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:53.604176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef74bb-00f6-4c81-bc2e-7d369691c159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:33:53.607276 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-85ef74bb-00f6-4c81-bc2e-7d369691c159 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:33:53.617109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:33:53.619066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:33:53.620282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 7/14] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:33:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:33:53.624490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef74bb-00f6-4c81-bc2e-7d369691c159 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:53.688889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 8/15] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:33:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:33:53.694872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974160ec-ff31-436a-9b9f-7fc20414b7f7 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:53.726805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef74bb-00f6-4c81-bc2e-7d369691c159 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:53.762072 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f03b77-7f4a-4fe3-874c-96d1485a30a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:53.765201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f03b77-7f4a-4fe3-874c-96d1485a30a9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:53.765201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f03b77-7f4a-4fe3-874c-96d1485a30a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:53.765201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f03b77-7f4a-4fe3-874c-96d1485a30a9 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:53.765201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f03b77-7f4a-4fe3-874c-96d1485a30a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:33:53.765201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f03b77-7f4a-4fe3-874c-96d1485a30a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:53.767683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f03b77-7f4a-4fe3-874c-96d1485a30a9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:53.778803 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 8/16] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:33:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:33:53.784864 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f03b77-7f4a-4fe3-874c-96d1485a30a9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:53.805416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974160ec-ff31-436a-9b9f-7fc20414b7f7 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:53.813190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:53.832249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:53.851590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c2c55-dce2-4a3b-af5d-51858c1bf053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:53.852699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c2c55-dce2-4a3b-af5d-51858c1bf053 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:53.853189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c2c55-dce2-4a3b-af5d-51858c1bf053 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:53.853704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c2c55-dce2-4a3b-af5d-51858c1bf053 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:53.866162 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c2c55-dce2-4a3b-af5d-51858c1bf053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:53.867661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c2c55-dce2-4a3b-af5d-51858c1bf053 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-fa3c2c55-dce2-4a3b-af5d-51858c1bf053 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:33:53.889488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:33:53.890887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:33:53.891661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:33:53.891661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:33:53.891661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:33:53.891661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:33:53.891661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 9/17] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:33:53.895226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f03b77-7f4a-4fe3-874c-96d1485a30a9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:53.896969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c2c55-dce2-4a3b-af5d-51858c1bf053 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:53.930099 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbb984-e805-4ab9-8318-d116f9370f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:53.930984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbb984-e805-4ab9-8318-d116f9370f42 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:53.931271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbb984-e805-4ab9-8318-d116f9370f42 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:53.931547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbb984-e805-4ab9-8318-d116f9370f42 None admin] PATH_INFO: `/v3/users/629381bc156b476a94122bdd0a64b60d` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:53.933124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbb984-e805-4ab9-8318-d116f9370f42 None admin] RBAC: Authorizing `identity:get_user(user_id=629381bc156b476a94122bdd0a64b60d)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:53.934617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbb984-e805-4ab9-8318-d116f9370f42 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:53.936476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 9/18] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:33:53 2022] GET /identity/v3/users/629381bc156b476a94122bdd0a64b60d => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:33:53.941970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbb984-e805-4ab9-8318-d116f9370f42 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:54.000924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c2c55-dce2-4a3b-af5d-51858c1bf053 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:54.036808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:54.037084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e09c21-115a-4faa-ba65-c793925497dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:54.037800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e09c21-115a-4faa-ba65-c793925497dc None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:54.038103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e09c21-115a-4faa-ba65-c793925497dc None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:54.038408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e09c21-115a-4faa-ba65-c793925497dc None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:54.039326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e09c21-115a-4faa-ba65-c793925497dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:33:54.039734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e09c21-115a-4faa-ba65-c793925497dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:54.041169 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e09c21-115a-4faa-ba65-c793925497dc None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:54.044524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbb984-e805-4ab9-8318-d116f9370f42 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:54.059008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 10/19] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:33:54.081883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae4206-ac6b-4f61-ad73-a98aae0e2153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:54.083036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae4206-ac6b-4f61-ad73-a98aae0e2153 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:54.083151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae4206-ac6b-4f61-ad73-a98aae0e2153 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:54.083396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae4206-ac6b-4f61-ad73-a98aae0e2153 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:54.084762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae4206-ac6b-4f61-ad73-a98aae0e2153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:54.086300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae4206-ac6b-4f61-ad73-a98aae0e2153 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:54.088048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 10/20] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:33:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:33:54.095356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e09c21-115a-4faa-ba65-c793925497dc None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:54.133894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:54.214007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e09c21-115a-4faa-ba65-c793925497dc None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:54.261273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28388ef8-d34b-4dac-9ea8-f37c80308290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:54.261909 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28388ef8-d34b-4dac-9ea8-f37c80308290 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:54.262198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28388ef8-d34b-4dac-9ea8-f37c80308290 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:54.262478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28388ef8-d34b-4dac-9ea8-f37c80308290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:54.263426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28388ef8-d34b-4dac-9ea8-f37c80308290 None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.domain.id=default, user.id=629381bc156b476a94122bdd0a64b60d) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:33:54.263744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28388ef8-d34b-4dac-9ea8-f37c80308290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:54.265861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28388ef8-d34b-4dac-9ea8-f37c80308290 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:54.273874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 11/21] 10.222.0.70 () {56 vars in 1322 bytes} [Tue Jul 19 20:33:54 2022] GET /identity/v3/role_assignments?user.id=629381bc156b476a94122bdd0a64b60d&scope.domain.id=default&role.id=85420c68300642aabfd730fb8fef8347 => generated 233 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:33:56.497400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27407-a73e-4a13-9bf0-b1910ca19cbc None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:56.498113 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27407-a73e-4a13-9bf0-b1910ca19cbc None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:56.498113 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b27407-a73e-4a13-9bf0-b1910ca19cbc None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:56.499886 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 11/22] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:33:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:33:56.506028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32500898-6eab-40f4-ab44-bca72d71b90e None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:56.506547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32500898-6eab-40f4-ab44-bca72d71b90e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:56.506646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32500898-6eab-40f4-ab44-bca72d71b90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:56.531294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-32500898-6eab-40f4-ab44-bca72d71b90e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:33:56.532542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-32500898-6eab-40f4-ab44-bca72d71b90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:33:56.638247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 12/23] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:33:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:33:56.701551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b27407-a73e-4a13-9bf0-b1910ca19cbc None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:56.859747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b27407-a73e-4a13-9bf0-b1910ca19cbc None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:56.904460 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff14367-5f1b-4bbf-99bf-c2e32eac2479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:56.905531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff14367-5f1b-4bbf-99bf-c2e32eac2479 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:56.905531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff14367-5f1b-4bbf-99bf-c2e32eac2479 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:56.905818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff14367-5f1b-4bbf-99bf-c2e32eac2479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:56.935891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff14367-5f1b-4bbf-99bf-c2e32eac2479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:56.938002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff14367-5f1b-4bbf-99bf-c2e32eac2479 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-4ff14367-5f1b-4bbf-99bf-c2e32eac2479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:33:56.964759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:33:56.965671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:33:56.966459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:33:56.966459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:33:56.966459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:33:56.966459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:33:56.966459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 12/24] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:33:56.978587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32500898-6eab-40f4-ab44-bca72d71b90e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:57.034807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:57.096157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d392887-5355-45f6-b961-3aac125c9c4e None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:57.096483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d392887-5355-45f6-b961-3aac125c9c4e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:57.096739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d392887-5355-45f6-b961-3aac125c9c4e None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:57.098223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 13/25] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:33:57.107600 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788a25e-827b-4cc8-9652-eae9bd6cf839 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:57.108103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788a25e-827b-4cc8-9652-eae9bd6cf839 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:57.109090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788a25e-827b-4cc8-9652-eae9bd6cf839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:57.145476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a788a25e-827b-4cc8-9652-eae9bd6cf839 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:33:57.146718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a788a25e-827b-4cc8-9652-eae9bd6cf839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:33:57.152470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32500898-6eab-40f4-ab44-bca72d71b90e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:57.168725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:57.216821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf2821-b583-47a4-b25b-fab6f0382bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:57.217536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2821-b583-47a4-b25b-fab6f0382bea None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:57.218402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2821-b583-47a4-b25b-fab6f0382bea None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:57.218685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2821-b583-47a4-b25b-fab6f0382bea None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:57.219443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf2821-b583-47a4-b25b-fab6f0382bea None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:33:57.219759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf2821-b583-47a4-b25b-fab6f0382bea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:57.221177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf2821-b583-47a4-b25b-fab6f0382bea None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:57.247572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 13/26] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:33:57.262856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a49e4-adf0-40cf-8852-dc278ca60c42 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:57.263161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a49e4-adf0-40cf-8852-dc278ca60c42 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:57.263435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a49e4-adf0-40cf-8852-dc278ca60c42 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:57.265691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 14/27] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:33:57 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:33:57.266329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a49e4-adf0-40cf-8852-dc278ca60c42 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:57.309223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 14/28] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:33:57.315126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199ae1b-1a17-4401-8adc-834ba51acb34 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:57.315416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199ae1b-1a17-4401-8adc-834ba51acb34 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:57.315704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199ae1b-1a17-4401-8adc-834ba51acb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:57.342977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0199ae1b-1a17-4401-8adc-834ba51acb34 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:33:57.344535 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0199ae1b-1a17-4401-8adc-834ba51acb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:33:57.404242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:57.422579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a49e4-adf0-40cf-8852-dc278ca60c42 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:57.441966 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 15/29] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:33:57.445268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023a7b-a180-45ce-816d-177d4a1d51bb None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:57.445810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023a7b-a180-45ce-816d-177d4a1d51bb None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:57.446274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023a7b-a180-45ce-816d-177d4a1d51bb None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:57.453123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 16/30] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:33:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:33:57.454428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05023a7b-a180-45ce-816d-177d4a1d51bb None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:57.469253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c1e93-1eed-4562-9c29-ce39c12451c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:57.470117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c1e93-1eed-4562-9c29-ce39c12451c8 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:57.470927 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c1e93-1eed-4562-9c29-ce39c12451c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:57.471320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c1e93-1eed-4562-9c29-ce39c12451c8 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:57.472486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c1e93-1eed-4562-9c29-ce39c12451c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:57.474344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c1e93-1eed-4562-9c29-ce39c12451c8 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:57.560413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:57.593164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05023a7b-a180-45ce-816d-177d4a1d51bb None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:57.632920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30c217-399f-4a86-9dd1-80af912d4d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:57.633795 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30c217-399f-4a86-9dd1-80af912d4d40 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:57.634216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30c217-399f-4a86-9dd1-80af912d4d40 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:57.634596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30c217-399f-4a86-9dd1-80af912d4d40 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:57.635489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30c217-399f-4a86-9dd1-80af912d4d40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:57.636973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30c217-399f-4a86-9dd1-80af912d4d40 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:57.648926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 15/31] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:33:57 2022] POST /identity/v3/projects => generated 301 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:33:57.651318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c6410-511b-4339-8e14-86438dfc92ca None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:57.652081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c6410-511b-4339-8e14-86438dfc92ca None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:57.652549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c6410-511b-4339-8e14-86438dfc92ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:57.680388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f19c6410-511b-4339-8e14-86438dfc92ca None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:33:57.682477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f19c6410-511b-4339-8e14-86438dfc92ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:33:57.766691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 16/32] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:33:57.773378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c6410-511b-4339-8e14-86438dfc92ca None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:57.788254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 17/33] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Jul 19 20:33:57 2022] POST /identity/v3/roles => generated 225 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:33:57.814771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30c217-399f-4a86-9dd1-80af912d4d40 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:57.932402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:58.004579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c6410-511b-4339-8e14-86438dfc92ca None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:58.010772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30c217-399f-4a86-9dd1-80af912d4d40 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:58.055238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3244f-e5ff-4fd4-b4e9-8956c707facb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:58.055929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3244f-e5ff-4fd4-b4e9-8956c707facb None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.056223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3244f-e5ff-4fd4-b4e9-8956c707facb None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.056507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3244f-e5ff-4fd4-b4e9-8956c707facb None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.057041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12db6d2-4b8a-4687-b6b6-aac3d0dbfe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:58.057282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3244f-e5ff-4fd4-b4e9-8956c707facb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:58.057977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12db6d2-4b8a-4687-b6b6-aac3d0dbfe9a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.058265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12db6d2-4b8a-4687-b6b6-aac3d0dbfe9a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.058947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12db6d2-4b8a-4687-b6b6-aac3d0dbfe9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.059091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3244f-e5ff-4fd4-b4e9-8956c707facb None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:58.065901 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:58.074123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12db6d2-4b8a-4687-b6b6-aac3d0dbfe9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:33:58.080990 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 18/34] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Jul 19 20:33:57 2022] POST /identity/v3/roles => generated 219 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-f12db6d2-4b8a-4687-b6b6-aac3d0dbfe9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:33:58.086738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:33:58.087739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:33:58.088612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 17/35] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:33:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:33:58.098658 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3244f-e5ff-4fd4-b4e9-8956c707facb None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:58.221885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3244f-e5ff-4fd4-b4e9-8956c707facb None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:58.268144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5665dd8-658e-402b-96c2-7f111e1df380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:58.269219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5665dd8-658e-402b-96c2-7f111e1df380 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.269741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5665dd8-658e-402b-96c2-7f111e1df380 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.270222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5665dd8-658e-402b-96c2-7f111e1df380 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.271244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5665dd8-658e-402b-96c2-7f111e1df380 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:33:58.271810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5665dd8-658e-402b-96c2-7f111e1df380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:58.273490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5665dd8-658e-402b-96c2-7f111e1df380 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:58.284097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 19/36] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:33:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:33:58.291019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12db6d2-4b8a-4687-b6b6-aac3d0dbfe9a None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:58.339066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3856b-a12b-45b6-8f6a-ac5715bd6e3d None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.339404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3856b-a12b-45b6-8f6a-ac5715bd6e3d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.339637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3856b-a12b-45b6-8f6a-ac5715bd6e3d None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.341038 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 20/37] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:33:58.347745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b228c-be12-464b-981f-20aa424e5048 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.348206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b228c-be12-464b-981f-20aa424e5048 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.348627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b228c-be12-464b-981f-20aa424e5048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.375737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-988b228c-be12-464b-981f-20aa424e5048 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:33:58.377774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-988b228c-be12-464b-981f-20aa424e5048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:33:58.430676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12db6d2-4b8a-4687-b6b6-aac3d0dbfe9a None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:58.446301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:58.464399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:58.465954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 21/38] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:33:58.467573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5112613-2255-4a98-8f2e-633bbbe64bb4 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.467948 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5112613-2255-4a98-8f2e-633bbbe64bb4 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.468310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5112613-2255-4a98-8f2e-633bbbe64bb4 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.470047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 22/39] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:33:58.470681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389e3ec-c9e9-4c7d-88b1-e2c113938ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:58.472075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389e3ec-c9e9-4c7d-88b1-e2c113938ec8 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.472564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389e3ec-c9e9-4c7d-88b1-e2c113938ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.473130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389e3ec-c9e9-4c7d-88b1-e2c113938ec8 None admin] PATH_INFO: `/v3/users/629381bc156b476a94122bdd0a64b60d` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.474918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389e3ec-c9e9-4c7d-88b1-e2c113938ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=629381bc156b476a94122bdd0a64b60d)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:58.475571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87609340-0ef6-4ec7-ba50-e891ac029b3a None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.475876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87609340-0ef6-4ec7-ba50-e891ac029b3a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.476167 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87609340-0ef6-4ec7-ba50-e891ac029b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.477157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389e3ec-c9e9-4c7d-88b1-e2c113938ec8 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:58.479116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 18/40] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:33:58 2022] GET /identity/v3/users/629381bc156b476a94122bdd0a64b60d => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:33:58.484843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389e3ec-c9e9-4c7d-88b1-e2c113938ec8 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:58.497735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-87609340-0ef6-4ec7-ba50-e891ac029b3a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:33:58.499581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-87609340-0ef6-4ec7-ba50-e891ac029b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:33:58.573044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 23/41] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:33:58.574914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87609340-0ef6-4ec7-ba50-e891ac029b3a None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:58.596098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389e3ec-c9e9-4c7d-88b1-e2c113938ec8 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:58.632814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c63257-2d7b-47c4-8ea9-1e0024d3038d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:58.633673 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c63257-2d7b-47c4-8ea9-1e0024d3038d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.633962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c63257-2d7b-47c4-8ea9-1e0024d3038d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.634244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c63257-2d7b-47c4-8ea9-1e0024d3038d None admin] PATH_INFO: `/v3/domains/default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.635501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c63257-2d7b-47c4-8ea9-1e0024d3038d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:58.636988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c63257-2d7b-47c4-8ea9-1e0024d3038d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:58.638710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 19/42] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:33:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:33:58.639554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c63257-2d7b-47c4-8ea9-1e0024d3038d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:58.696970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87609340-0ef6-4ec7-ba50-e891ac029b3a None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:58.710330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:58.717438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:58.737274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b6358-ed63-4811-9efe-a4a8fdfc9e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:58.737918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b6358-ed63-4811-9efe-a4a8fdfc9e1e None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.738213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b6358-ed63-4811-9efe-a4a8fdfc9e1e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.738504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b6358-ed63-4811-9efe-a4a8fdfc9e1e None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.739284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b6358-ed63-4811-9efe-a4a8fdfc9e1e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:58.740739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b6358-ed63-4811-9efe-a4a8fdfc9e1e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:58.756194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 24/43] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Jul 19 20:33:58 2022] POST /identity/v3/roles => generated 223 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:33:58.757039 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b6358-ed63-4811-9efe-a4a8fdfc9e1e None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:58.860848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c63257-2d7b-47c4-8ea9-1e0024d3038d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:58.895210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b6358-ed63-4811-9efe-a4a8fdfc9e1e None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:58.901239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff52b92-0aaa-4e9d-a581-3e3d86ac9558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:58.902110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff52b92-0aaa-4e9d-a581-3e3d86ac9558 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.902404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff52b92-0aaa-4e9d-a581-3e3d86ac9558 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.902687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff52b92-0aaa-4e9d-a581-3e3d86ac9558 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.904654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff52b92-0aaa-4e9d-a581-3e3d86ac9558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:58.907195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff52b92-0aaa-4e9d-a581-3e3d86ac9558 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:58.909401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 20/44] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:33:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:33:58.916516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff52b92-0aaa-4e9d-a581-3e3d86ac9558 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:33:58.937757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3425773-496b-494c-a42b-e481a9407076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:58.938651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3425773-496b-494c-a42b-e481a9407076 None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:58.938940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3425773-496b-494c-a42b-e481a9407076 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:58.939224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3425773-496b-494c-a42b-e481a9407076 None admin] PATH_INFO: `/v3/domains/default/users/629381bc156b476a94122bdd0a64b60d/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:58.941315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3425773-496b-494c-a42b-e481a9407076 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=629381bc156b476a94122bdd0a64b60d, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:58.942827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3425773-496b-494c-a42b-e481a9407076 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:58.959127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 25/45] 10.222.0.70 () {58 vars in 1361 bytes} [Tue Jul 19 20:33:58 2022] PUT /identity/v3/domains/default/users/629381bc156b476a94122bdd0a64b60d/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:33:59.038227 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff52b92-0aaa-4e9d-a581-3e3d86ac9558 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:33:59.079238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75038e-bac2-4076-8f93-eae1d4ac8a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:33:59.079927 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75038e-bac2-4076-8f93-eae1d4ac8a54 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:33:59.080214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75038e-bac2-4076-8f93-eae1d4ac8a54 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:33:59.080490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75038e-bac2-4076-8f93-eae1d4ac8a54 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:33:59.081225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75038e-bac2-4076-8f93-eae1d4ac8a54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:33:59.083365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75038e-bac2-4076-8f93-eae1d4ac8a54 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:33:59.087415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:33:59.120043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 21/46] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:33:58 2022] POST /identity/v3/projects => generated 312 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:34:01.033839 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc39567-dfa4-4398-aa8c-3b782210209e None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:01.034371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc39567-dfa4-4398-aa8c-3b782210209e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:01.034371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc39567-dfa4-4398-aa8c-3b782210209e None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:01.035774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 26/47] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:01.040979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1146d06-1363-45ad-91d0-885913489cab None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:01.041441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1146d06-1363-45ad-91d0-885913489cab None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:01.041607 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1146d06-1363-45ad-91d0-885913489cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:01.060002 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e1146d06-1363-45ad-91d0-885913489cab None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:01.061253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e1146d06-1363-45ad-91d0-885913489cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:01.136375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 22/48] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:01.160206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc39567-dfa4-4398-aa8c-3b782210209e None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:01.178225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f965a-8a6c-45d4-9c38-3b1d2ff88d88 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:01.178523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f965a-8a6c-45d4-9c38-3b1d2ff88d88 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:01.178812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f965a-8a6c-45d4-9c38-3b1d2ff88d88 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:01.180210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 23/49] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:34:01.185390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a39904-b57c-4c66-99b4-60fe47a92718 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:01.185695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a39904-b57c-4c66-99b4-60fe47a92718 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:01.185985 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a39904-b57c-4c66-99b4-60fe47a92718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:01.200393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:01.203214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-38a39904-b57c-4c66-99b4-60fe47a92718 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:01.204675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-38a39904-b57c-4c66-99b4-60fe47a92718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:01.270893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc39567-dfa4-4398-aa8c-3b782210209e None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:01.275807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 24/50] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:01.299516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a39904-b57c-4c66-99b4-60fe47a92718 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:01.306812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c933e8-f2da-45c6-abab-40bc7cd2a0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:01.307625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c933e8-f2da-45c6-abab-40bc7cd2a0a8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:01.307910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c933e8-f2da-45c6-abab-40bc7cd2a0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:01.308190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c933e8-f2da-45c6-abab-40bc7cd2a0a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:01.319967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c933e8-f2da-45c6-abab-40bc7cd2a0a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-c3c933e8-f2da-45c6-abab-40bc7cd2a0a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:01.328576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:01.329477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:01.330238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 27/51] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:34:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:01.335626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c933e8-f2da-45c6-abab-40bc7cd2a0a8 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:01.347881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:01.412496 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a39904-b57c-4c66-99b4-60fe47a92718 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:01.441891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:01.449205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c933e8-f2da-45c6-abab-40bc7cd2a0a8 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:01.450378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85361e11-ab1a-48f7-be37-11a59d512e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:01.451189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85361e11-ab1a-48f7-be37-11a59d512e86 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:01.451476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85361e11-ab1a-48f7-be37-11a59d512e86 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:01.451755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85361e11-ab1a-48f7-be37-11a59d512e86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:01.453521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85361e11-ab1a-48f7-be37-11a59d512e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:01.454978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85361e11-ab1a-48f7-be37-11a59d512e86 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:01.456648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 25/52] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:01.462268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85361e11-ab1a-48f7-be37-11a59d512e86 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:01.485171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d515c-356f-4a1c-9d33-77c116ffe83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:01.485885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d515c-356f-4a1c-9d33-77c116ffe83f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:01.486172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d515c-356f-4a1c-9d33-77c116ffe83f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:01.486450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d515c-356f-4a1c-9d33-77c116ffe83f None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:01.487212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d515c-356f-4a1c-9d33-77c116ffe83f None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:01.487521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d515c-356f-4a1c-9d33-77c116ffe83f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:01.489006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d515c-356f-4a1c-9d33-77c116ffe83f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:01.497693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 28/53] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:01.503371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d515c-356f-4a1c-9d33-77c116ffe83f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:01.576019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85361e11-ab1a-48f7-be37-11a59d512e86 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:01.594073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:01.616594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40fc8c-7cc7-41af-a5a3-074bdb3a1284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:01.617272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40fc8c-7cc7-41af-a5a3-074bdb3a1284 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:01.617589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40fc8c-7cc7-41af-a5a3-074bdb3a1284 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:01.617873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40fc8c-7cc7-41af-a5a3-074bdb3a1284 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:01.618598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40fc8c-7cc7-41af-a5a3-074bdb3a1284 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:01.620076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40fc8c-7cc7-41af-a5a3-074bdb3a1284 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:01.648005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 26/54] 10.222.0.70 () {60 vars in 1140 bytes} [Tue Jul 19 20:34:01 2022] POST /identity/v3/projects => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:34:01.724805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d515c-356f-4a1c-9d33-77c116ffe83f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:01.762329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cd2cc-cecd-4a67-a47b-ac7dc73e1d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:01.763234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cd2cc-cecd-4a67-a47b-ac7dc73e1d82 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:01.763521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cd2cc-cecd-4a67-a47b-ac7dc73e1d82 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:01.763799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cd2cc-cecd-4a67-a47b-ac7dc73e1d82 None admin] PATH_INFO: `/v3/users/629381bc156b476a94122bdd0a64b60d` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:01.765736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875cd2cc-cecd-4a67-a47b-ac7dc73e1d82 None admin] RBAC: Authorizing `identity:get_user(user_id=629381bc156b476a94122bdd0a64b60d)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:01.767212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875cd2cc-cecd-4a67-a47b-ac7dc73e1d82 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:01.768936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 29/55] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:01 2022] GET /identity/v3/users/629381bc156b476a94122bdd0a64b60d => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:01.774591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40fc8c-7cc7-41af-a5a3-074bdb3a1284 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:01.887248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40fc8c-7cc7-41af-a5a3-074bdb3a1284 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:01.921871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403e98c-c662-417e-820f-ca038b2f2727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:01.922690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403e98c-c662-417e-820f-ca038b2f2727 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:01.922980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403e98c-c662-417e-820f-ca038b2f2727 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:01.923262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403e98c-c662-417e-820f-ca038b2f2727 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:01.924485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403e98c-c662-417e-820f-ca038b2f2727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:01.926001 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403e98c-c662-417e-820f-ca038b2f2727 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:01.927900 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 27/56] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:01.934293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cd2cc-cecd-4a67-a47b-ac7dc73e1d82 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:01.963096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:02.040193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cd2cc-cecd-4a67-a47b-ac7dc73e1d82 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:02.077340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6e8b3-3214-4711-8b39-68407c5cb7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:02.078059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e8b3-3214-4711-8b39-68407c5cb7a7 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:02.078354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e8b3-3214-4711-8b39-68407c5cb7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:02.078639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e8b3-3214-4711-8b39-68407c5cb7a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:02.079499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6e8b3-3214-4711-8b39-68407c5cb7a7 None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.domain.id=default, user.id=629381bc156b476a94122bdd0a64b60d) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:02.079816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6e8b3-3214-4711-8b39-68407c5cb7a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:02.081269 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6e8b3-3214-4711-8b39-68407c5cb7a7 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:02.090734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 30/57] 10.222.0.70 () {56 vars in 1322 bytes} [Tue Jul 19 20:34:01 2022] GET /identity/v3/role_assignments?user.id=629381bc156b476a94122bdd0a64b60d&scope.domain.id=default&role.id=85420c68300642aabfd730fb8fef8347 => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:03.731033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9b9c-be8f-4b1c-8aba-da9d759ba08b None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:03.731553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9b9c-be8f-4b1c-8aba-da9d759ba08b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:03.731553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9b9c-be8f-4b1c-8aba-da9d759ba08b None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:03.732902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 28/58] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:03.741786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d906a0-c0c9-406c-abc3-837f7196e1fa None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:03.742285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d906a0-c0c9-406c-abc3-837f7196e1fa None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:03.742780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d906a0-c0c9-406c-abc3-837f7196e1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:03.769334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-98d906a0-c0c9-406c-abc3-837f7196e1fa None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:03.771422 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-98d906a0-c0c9-406c-abc3-837f7196e1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:03.870793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 31/59] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:03.896114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b9b9c-be8f-4b1c-8aba-da9d759ba08b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:03.935631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:04.009470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b9b9c-be8f-4b1c-8aba-da9d759ba08b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:04.042667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d1d3b-e63a-45de-b874-4d1f7c428432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:04.043493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d1d3b-e63a-45de-b874-4d1f7c428432 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:04.043771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d1d3b-e63a-45de-b874-4d1f7c428432 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:04.044057 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d1d3b-e63a-45de-b874-4d1f7c428432 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:04.045244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d1d3b-e63a-45de-b874-4d1f7c428432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:04.047324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d1d3b-e63a-45de-b874-4d1f7c428432 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:04.048975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 29/60] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:04.054570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d906a0-c0c9-406c-abc3-837f7196e1fa None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:04.075816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:04.152581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d906a0-c0c9-406c-abc3-837f7196e1fa None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:04.186390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2f8df-8764-4009-9ed9-10b018647021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:04.187056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2f8df-8764-4009-9ed9-10b018647021 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:04.187339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2f8df-8764-4009-9ed9-10b018647021 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:04.187617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2f8df-8764-4009-9ed9-10b018647021 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:04.188338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2f8df-8764-4009-9ed9-10b018647021 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:04.189810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2f8df-8764-4009-9ed9-10b018647021 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:04.216663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 32/61] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:34:04 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:34:07.476240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251375cd-1280-4daf-b258-f065f1c611ac None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:07.477113 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251375cd-1280-4daf-b258-f065f1c611ac None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:07.477579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251375cd-1280-4daf-b258-f065f1c611ac None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:07.479102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 30/62] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:07.484557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686316a-f25f-40d3-a00e-f22958080fb5 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:07.485327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686316a-f25f-40d3-a00e-f22958080fb5 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:07.485773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686316a-f25f-40d3-a00e-f22958080fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:07.508123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3834c-7fe4-4115-b799-9bde8daf0262 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:07.508740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3834c-7fe4-4115-b799-9bde8daf0262 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:07.509242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3834c-7fe4-4115-b799-9bde8daf0262 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:07.510418 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-7686316a-f25f-40d3-a00e-f22958080fb5 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:07.511107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 31/63] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:07.511930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7686316a-f25f-40d3-a00e-f22958080fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:07.517530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157fdba-6e23-4b4f-ac23-c73338fafbbc None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:07.518136 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157fdba-6e23-4b4f-ac23-c73338fafbbc None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:07.518616 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157fdba-6e23-4b4f-ac23-c73338fafbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:07.536963 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-6157fdba-6e23-4b4f-ac23-c73338fafbbc None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:07.538534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6157fdba-6e23-4b4f-ac23-c73338fafbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:07.611674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 32/64] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:07.635770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157fdba-6e23-4b4f-ac23-c73338fafbbc None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:07.682942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 33/65] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:07.689724 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:07.730858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686316a-f25f-40d3-a00e-f22958080fb5 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:07.746143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:07.747645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157fdba-6e23-4b4f-ac23-c73338fafbbc None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:07.791483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5d044-a14e-4f80-a10f-bf64af2631ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:07.792857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5d044-a14e-4f80-a10f-bf64af2631ba None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:07.794141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5d044-a14e-4f80-a10f-bf64af2631ba None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:07.794642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5d044-a14e-4f80-a10f-bf64af2631ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:07.808814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5d044-a14e-4f80-a10f-bf64af2631ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-44b5d044-a14e-4f80-a10f-bf64af2631ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:07.819838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:07.820727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:07.821524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:07.827700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 33/66] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:07.827700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0046b-81ad-44e4-a5e2-e56dd2843830 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:07.827700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0046b-81ad-44e4-a5e2-e56dd2843830 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:07.827700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0046b-81ad-44e4-a5e2-e56dd2843830 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:07.827700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 34/67] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:07.831152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0046b-81ad-44e4-a5e2-e56dd2843830 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:07.878398 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686316a-f25f-40d3-a00e-f22958080fb5 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:07.919986 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1b109-79fc-4a2d-9ec9-2053fdb3d48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:07.920748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1b109-79fc-4a2d-9ec9-2053fdb3d48d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:07.921038 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1b109-79fc-4a2d-9ec9-2053fdb3d48d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:07.921318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1b109-79fc-4a2d-9ec9-2053fdb3d48d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:07.933292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1b109-79fc-4a2d-9ec9-2053fdb3d48d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-95a1b109-79fc-4a2d-9ec9-2053fdb3d48d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:07.941760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:07.942603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:07.943354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 34/68] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:34:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:07.944438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ef560-a25f-4bcd-b8b5-b31366daa36e None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:07.944528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ef560-a25f-4bcd-b8b5-b31366daa36e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:07.944753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ef560-a25f-4bcd-b8b5-b31366daa36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:07.948301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:07.962511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0046b-81ad-44e4-a5e2-e56dd2843830 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:07.965604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-626ef560-a25f-4bcd-b8b5-b31366daa36e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:07.966583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-626ef560-a25f-4bcd-b8b5-b31366daa36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:08.001272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1a8a0-8135-4dd2-83e3-3b4c1b21b146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:08.001988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1a8a0-8135-4dd2-83e3-3b4c1b21b146 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.002282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1a8a0-8135-4dd2-83e3-3b4c1b21b146 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.002572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1a8a0-8135-4dd2-83e3-3b4c1b21b146 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.003349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1a8a0-8135-4dd2-83e3-3b4c1b21b146 None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:08.003664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1a8a0-8135-4dd2-83e3-3b4c1b21b146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:08.005060 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1a8a0-8135-4dd2-83e3-3b4c1b21b146 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:08.024534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 35/69] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:08.024534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5823f-5b24-414e-b95d-cc06ce5696e8 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.024534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5823f-5b24-414e-b95d-cc06ce5696e8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.024534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5823f-5b24-414e-b95d-cc06ce5696e8 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.024534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 36/70] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:08.024951 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5823f-5b24-414e-b95d-cc06ce5696e8 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:08.052205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:08.062807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 35/71] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:08.063483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ef560-a25f-4bcd-b8b5-b31366daa36e None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:08.152018 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5823f-5b24-414e-b95d-cc06ce5696e8 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:08.175477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ef560-a25f-4bcd-b8b5-b31366daa36e None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:08.192187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f11b5-5ff2-4e43-8e68-a29f0420ac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:08.192874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f11b5-5ff2-4e43-8e68-a29f0420ac08 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.193164 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f11b5-5ff2-4e43-8e68-a29f0420ac08 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.193488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f11b5-5ff2-4e43-8e68-a29f0420ac08 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.194258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f11b5-5ff2-4e43-8e68-a29f0420ac08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:08.194705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f11b5-5ff2-4e43-8e68-a29f0420ac08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:08.195980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f11b5-5ff2-4e43-8e68-a29f0420ac08 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:08.205380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 37/72] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:08.206614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254c395-959b-4355-ba72-9affb662f986 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.206917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254c395-959b-4355-ba72-9affb662f986 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.207206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254c395-959b-4355-ba72-9affb662f986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.219734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed0e1c-f009-4925-b61d-8f0ea421f4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:08.220693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed0e1c-f009-4925-b61d-8f0ea421f4d4 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.220981 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed0e1c-f009-4925-b61d-8f0ea421f4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.221260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed0e1c-f009-4925-b61d-8f0ea421f4d4 None admin] PATH_INFO: `/v3/users/f9bf4be1c5e94c578c2e02b627c30a56` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.225257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-5254c395-959b-4355-ba72-9affb662f986 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:08.226511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5254c395-959b-4355-ba72-9affb662f986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:08.238956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed0e1c-f009-4925-b61d-8f0ea421f4d4 None admin] RBAC: Authorizing `identity:get_user(user_id=f9bf4be1c5e94c578c2e02b627c30a56)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:08.239778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:08.240597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed0e1c-f009-4925-b61d-8f0ea421f4d4 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:08.250745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 36/73] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:08 2022] GET /identity/v3/users/f9bf4be1c5e94c578c2e02b627c30a56 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:08.250745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed0e1c-f009-4925-b61d-8f0ea421f4d4 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:08.311614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 38/74] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:08.313106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254c395-959b-4355-ba72-9affb662f986 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:08.334443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:08.409729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed0e1c-f009-4925-b61d-8f0ea421f4d4 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:08.427785 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254c395-959b-4355-ba72-9affb662f986 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:08.449752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a27d8-ce3e-424e-a7ac-86444495bf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:08.450587 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a27d8-ce3e-424e-a7ac-86444495bf41 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.450871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a27d8-ce3e-424e-a7ac-86444495bf41 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.451143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a27d8-ce3e-424e-a7ac-86444495bf41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.462572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a27d8-ce3e-424e-a7ac-86444495bf41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:08.465310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b6a8a-bbed-4ded-8525-2b1d9d25eac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:08.466226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b6a8a-bbed-4ded-8525-2b1d9d25eac2 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.466516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b6a8a-bbed-4ded-8525-2b1d9d25eac2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.466798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b6a8a-bbed-4ded-8525-2b1d9d25eac2 None admin] PATH_INFO: `/v3/users/629381bc156b476a94122bdd0a64b60d` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.468533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b6a8a-bbed-4ded-8525-2b1d9d25eac2 None admin] RBAC: Authorizing `identity:get_user(user_id=629381bc156b476a94122bdd0a64b60d)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:08.470183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b6a8a-bbed-4ded-8525-2b1d9d25eac2 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-0c9a27d8-ce3e-424e-a7ac-86444495bf41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:08.471734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:08.472748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:08.473766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:08.474361 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 39/75] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:08 2022] GET /identity/v3/users/629381bc156b476a94122bdd0a64b60d => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:08.474361 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b6a8a-bbed-4ded-8525-2b1d9d25eac2 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:08.474476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 37/76] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:08.474476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a27d8-ce3e-424e-a7ac-86444495bf41 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:08.493139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:08.582533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b6a8a-bbed-4ded-8525-2b1d9d25eac2 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:08.583940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a27d8-ce3e-424e-a7ac-86444495bf41 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:08.602983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:08.618910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50794bcc-9790-493b-ba7e-9d1b0d519257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:08.619809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50794bcc-9790-493b-ba7e-9d1b0d519257 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.620095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50794bcc-9790-493b-ba7e-9d1b0d519257 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.620379 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50794bcc-9790-493b-ba7e-9d1b0d519257 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.625427 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b56a5-4686-4294-817e-1931a9db73b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:08.626241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b56a5-4686-4294-817e-1931a9db73b7 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.626529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b56a5-4686-4294-817e-1931a9db73b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.626810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b56a5-4686-4294-817e-1931a9db73b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.637380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b56a5-4686-4294-817e-1931a9db73b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-2d9b56a5-4686-4294-817e-1931a9db73b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:08.646210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:08.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:08.648430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 38/77] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:08.649073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555cf0e-bb2e-455a-bcf4-6a243f13420f None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.649073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555cf0e-bb2e-455a-bcf4-6a243f13420f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.649196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555cf0e-bb2e-455a-bcf4-6a243f13420f None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.650499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 39/78] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:08.653577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555cf0e-bb2e-455a-bcf4-6a243f13420f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:08.695081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50794bcc-9790-493b-ba7e-9d1b0d519257 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:08.696481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50794bcc-9790-493b-ba7e-9d1b0d519257 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:08.711132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:08.761501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555cf0e-bb2e-455a-bcf4-6a243f13420f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:08.797181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3a697-4cc8-4cb3-9a66-98f970f38bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:08.797905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3a697-4cc8-4cb3-9a66-98f970f38bd5 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.798197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3a697-4cc8-4cb3-9a66-98f970f38bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.798482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3a697-4cc8-4cb3-9a66-98f970f38bd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.815902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 40/79] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:08.816761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50794bcc-9790-493b-ba7e-9d1b0d519257 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:08.848025 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:08.873090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3a697-4cc8-4cb3-9a66-98f970f38bd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:08.874618 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3a697-4cc8-4cb3-9a66-98f970f38bd5 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:08.920508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50794bcc-9790-493b-ba7e-9d1b0d519257 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:08.935476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:08.959154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a6a8d-35dd-4a6e-ba53-e250c1ceaf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:08.959757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a6a8d-35dd-4a6e-ba53-e250c1ceaf86 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.960050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a6a8d-35dd-4a6e-ba53-e250c1ceaf86 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.960332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a6a8d-35dd-4a6e-ba53-e250c1ceaf86 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.961096 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a6a8d-35dd-4a6e-ba53-e250c1ceaf86 None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:08.961446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a6a8d-35dd-4a6e-ba53-e250c1ceaf86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:08.962844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a6a8d-35dd-4a6e-ba53-e250c1ceaf86 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:08.972154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 41/80] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:08.973053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a6a8d-35dd-4a6e-ba53-e250c1ceaf86 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:08.976769 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 40/81] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:08.978736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e57af-28ca-44b0-805d-4088053b34ec None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:08.979040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e57af-28ca-44b0-805d-4088053b34ec None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:08.979616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e57af-28ca-44b0-805d-4088053b34ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:08.996391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-7b7e57af-28ca-44b0-805d-4088053b34ec None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:08.997892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7b7e57af-28ca-44b0-805d-4088053b34ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:09.064624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 41/82] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:09.065516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e57af-28ca-44b0-805d-4088053b34ec None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:09.068747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:09.078210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a6a8d-35dd-4a6e-ba53-e250c1ceaf86 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:09.114958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eb6a0-4ddd-4b12-b35f-0eb307f66a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:09.115643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eb6a0-4ddd-4b12-b35f-0eb307f66a3c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:09.115927 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eb6a0-4ddd-4b12-b35f-0eb307f66a3c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:09.116232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eb6a0-4ddd-4b12-b35f-0eb307f66a3c None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:09.116986 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eb6a0-4ddd-4b12-b35f-0eb307f66a3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:09.117296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eb6a0-4ddd-4b12-b35f-0eb307f66a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:09.118692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eb6a0-4ddd-4b12-b35f-0eb307f66a3c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:09.127203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 42/83] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:09.128008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eb6a0-4ddd-4b12-b35f-0eb307f66a3c None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:09.171079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e57af-28ca-44b0-805d-4088053b34ec None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:09.174727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:09.206054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3aeff6-e50f-446d-a4b8-fbe90706a8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:09.207006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3aeff6-e50f-446d-a4b8-fbe90706a8c4 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:09.207211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3aeff6-e50f-446d-a4b8-fbe90706a8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:09.207487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3aeff6-e50f-446d-a4b8-fbe90706a8c4 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:09.208741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3aeff6-e50f-446d-a4b8-fbe90706a8c4 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:09.210325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3aeff6-e50f-446d-a4b8-fbe90706a8c4 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:09.212023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 42/84] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:09 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:09.212833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3aeff6-e50f-446d-a4b8-fbe90706a8c4 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:09.237710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eb6a0-4ddd-4b12-b35f-0eb307f66a3c None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:09.274880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c434f-e342-4419-ab80-412c25737e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:09.275866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c434f-e342-4419-ab80-412c25737e75 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:09.276159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c434f-e342-4419-ab80-412c25737e75 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:09.276439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c434f-e342-4419-ab80-412c25737e75 None admin] PATH_INFO: `/v3/users/f9bf4be1c5e94c578c2e02b627c30a56` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:09.277707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c434f-e342-4419-ab80-412c25737e75 None admin] RBAC: Authorizing `identity:get_user(user_id=f9bf4be1c5e94c578c2e02b627c30a56)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:09.279149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c434f-e342-4419-ab80-412c25737e75 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:09.281315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 43/85] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:09 2022] GET /identity/v3/users/f9bf4be1c5e94c578c2e02b627c30a56 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:09.286329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c434f-e342-4419-ab80-412c25737e75 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:09.293597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:09.324010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3aeff6-e50f-446d-a4b8-fbe90706a8c4 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:09.358561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1631be-4e1d-4ab3-9d6b-5d46d4742d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:09.359441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1631be-4e1d-4ab3-9d6b-5d46d4742d29 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:09.359726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1631be-4e1d-4ab3-9d6b-5d46d4742d29 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:09.360005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1631be-4e1d-4ab3-9d6b-5d46d4742d29 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:09.361226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1631be-4e1d-4ab3-9d6b-5d46d4742d29 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:09.362744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1631be-4e1d-4ab3-9d6b-5d46d4742d29 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:09.364558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 43/86] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:09 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:09.365296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1631be-4e1d-4ab3-9d6b-5d46d4742d29 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:09.389601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c434f-e342-4419-ab80-412c25737e75 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:09.397554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:09.426172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60daff-5404-4afc-8961-fa2fa715bced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:09.426978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60daff-5404-4afc-8961-fa2fa715bced None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:09.427271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60daff-5404-4afc-8961-fa2fa715bced None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:09.427556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60daff-5404-4afc-8961-fa2fa715bced None admin] PATH_INFO: `/v3/domains/default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:09.428742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60daff-5404-4afc-8961-fa2fa715bced None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:09.430227 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60daff-5404-4afc-8961-fa2fa715bced None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:09.431859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 44/87] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:09.432645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60daff-5404-4afc-8961-fa2fa715bced None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:09.477686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1631be-4e1d-4ab3-9d6b-5d46d4742d29 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:09.512806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36df3a-24cb-4b7e-a48c-4663b8a98f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:09.513719 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36df3a-24cb-4b7e-a48c-4663b8a98f56 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:09.514086 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36df3a-24cb-4b7e-a48c-4663b8a98f56 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:09.514809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36df3a-24cb-4b7e-a48c-4663b8a98f56 None admin] PATH_INFO: `/v3/users/f9bf4be1c5e94c578c2e02b627c30a56` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:09.516034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36df3a-24cb-4b7e-a48c-4663b8a98f56 None admin] RBAC: Authorizing `identity:get_user(user_id=f9bf4be1c5e94c578c2e02b627c30a56)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:09.517530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36df3a-24cb-4b7e-a48c-4663b8a98f56 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:09.519189 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 44/88] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:09 2022] GET /identity/v3/users/f9bf4be1c5e94c578c2e02b627c30a56 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:09.523885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36df3a-24cb-4b7e-a48c-4663b8a98f56 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:09.532320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:09.536993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60daff-5404-4afc-8961-fa2fa715bced None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:09.575184 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255476c3-aea4-4117-9e4e-aad36011af5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:09.575846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255476c3-aea4-4117-9e4e-aad36011af5b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:09.576130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255476c3-aea4-4117-9e4e-aad36011af5b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:09.576409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255476c3-aea4-4117-9e4e-aad36011af5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:09.577293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255476c3-aea4-4117-9e4e-aad36011af5b None admin] RBAC: Adding query filter params (role.id=d94de411ec0a40be92c04950e6edcac7, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user.id=f9bf4be1c5e94c578c2e02b627c30a56) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:09.577637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255476c3-aea4-4117-9e4e-aad36011af5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:09.579086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255476c3-aea4-4117-9e4e-aad36011af5b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:09.586603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 45/89] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:09 2022] GET /identity/v3/role_assignments?user.id=f9bf4be1c5e94c578c2e02b627c30a56&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:09.587565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255476c3-aea4-4117-9e4e-aad36011af5b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:09.639525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:09.642745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36df3a-24cb-4b7e-a48c-4663b8a98f56 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:09.689536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7fe26-14df-472e-a634-9c8d4aa86919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:09.690560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7fe26-14df-472e-a634-9c8d4aa86919 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:09.691018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7fe26-14df-472e-a634-9c8d4aa86919 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:09.691327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7fe26-14df-472e-a634-9c8d4aa86919 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:09.714451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255476c3-aea4-4117-9e4e-aad36011af5b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:09.757571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f635f-ae25-4b49-9e76-7f55a0701a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:09.758204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f635f-ae25-4b49-9e76-7f55a0701a66 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:09.758469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f635f-ae25-4b49-9e76-7f55a0701a66 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:09.758775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f635f-ae25-4b49-9e76-7f55a0701a66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:09.759609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f635f-ae25-4b49-9e76-7f55a0701a66 None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user.id=629381bc156b476a94122bdd0a64b60d) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:09.759930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f635f-ae25-4b49-9e76-7f55a0701a66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:09.761429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f635f-ae25-4b49-9e76-7f55a0701a66 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:09.777414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 46/90] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:09 2022] GET /identity/v3/role_assignments?user.id=629381bc156b476a94122bdd0a64b60d&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=85420c68300642aabfd730fb8fef8347 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:09.777414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f635f-ae25-4b49-9e76-7f55a0701a66 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:09.787837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7fe26-14df-472e-a634-9c8d4aa86919 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:09.789626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7fe26-14df-472e-a634-9c8d4aa86919 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:09.802057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:09.928996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f635f-ae25-4b49-9e76-7f55a0701a66 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:09.931467 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:09.937095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 45/91] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:09.937095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7fe26-14df-472e-a634-9c8d4aa86919 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:09.980640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb10ad-47ba-42f1-869b-8963fdbce40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:09.981339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb10ad-47ba-42f1-869b-8963fdbce40f None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:09.981658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb10ad-47ba-42f1-869b-8963fdbce40f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:09.981944 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb10ad-47ba-42f1-869b-8963fdbce40f None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:09.982683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb10ad-47ba-42f1-869b-8963fdbce40f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:09.985934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb10ad-47ba-42f1-869b-8963fdbce40f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:10.025496 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 47/92] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:34:09 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:34:10.025496 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb10ad-47ba-42f1-869b-8963fdbce40f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:10.111595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7fe26-14df-472e-a634-9c8d4aa86919 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:10.122668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:10.163746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2899e3d-5862-4818-870e-09ec15a4d868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:10.164942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2899e3d-5862-4818-870e-09ec15a4d868 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:10.168244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2899e3d-5862-4818-870e-09ec15a4d868 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:10.168858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2899e3d-5862-4818-870e-09ec15a4d868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:10.236247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb10ad-47ba-42f1-869b-8963fdbce40f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:10.288405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2899e3d-5862-4818-870e-09ec15a4d868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:10.291106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2899e3d-5862-4818-870e-09ec15a4d868 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:10.301072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4245b-c909-4604-a563-a80fa42b2ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:10.302160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4245b-c909-4604-a563-a80fa42b2ce0 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:10.303287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4245b-c909-4604-a563-a80fa42b2ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:10.303741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4245b-c909-4604-a563-a80fa42b2ce0 None admin] PATH_INFO: `/v3/projects/88dbcaf6b71a49eeb0297a64781368f4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:10.306208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4245b-c909-4604-a563-a80fa42b2ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=88dbcaf6b71a49eeb0297a64781368f4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:10.309344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4245b-c909-4604-a563-a80fa42b2ce0 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:10.313908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 48/93] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:10 2022] GET /identity/v3/projects/88dbcaf6b71a49eeb0297a64781368f4 => generated 312 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:10.322589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4245b-c909-4604-a563-a80fa42b2ce0 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:10.359249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:10.435278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 46/94] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:10.441812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2899e3d-5862-4818-870e-09ec15a4d868 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:10.455418 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:10.456876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4245b-c909-4604-a563-a80fa42b2ce0 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:10.501776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e6a18-c73a-4deb-a452-e21bd7fa7358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:10.503598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e6a18-c73a-4deb-a452-e21bd7fa7358 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:10.504118 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e6a18-c73a-4deb-a452-e21bd7fa7358 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:10.504794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e6a18-c73a-4deb-a452-e21bd7fa7358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:10.505918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e6a18-c73a-4deb-a452-e21bd7fa7358 None admin] RBAC: Adding query filter params (role.id=d94de411ec0a40be92c04950e6edcac7, scope.project.id=88dbcaf6b71a49eeb0297a64781368f4, user.id=f9bf4be1c5e94c578c2e02b627c30a56) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:10.506446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e6a18-c73a-4deb-a452-e21bd7fa7358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:10.508115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e6a18-c73a-4deb-a452-e21bd7fa7358 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:10.517435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 49/95] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:10 2022] GET /identity/v3/role_assignments?user.id=f9bf4be1c5e94c578c2e02b627c30a56&scope.project.id=88dbcaf6b71a49eeb0297a64781368f4&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:10.574033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2899e3d-5862-4818-870e-09ec15a4d868 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:10.616414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321790b5-0a96-4c1e-9b2f-52c12f1328ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:10.617750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321790b5-0a96-4c1e-9b2f-52c12f1328ab None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:10.618266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321790b5-0a96-4c1e-9b2f-52c12f1328ab None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:10.618746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321790b5-0a96-4c1e-9b2f-52c12f1328ab None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:10.620258 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321790b5-0a96-4c1e-9b2f-52c12f1328ab None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:10.622073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321790b5-0a96-4c1e-9b2f-52c12f1328ab None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:10.625034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 47/96] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:10 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:10.631748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e6a18-c73a-4deb-a452-e21bd7fa7358 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:10.743603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:10.757530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e6a18-c73a-4deb-a452-e21bd7fa7358 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:10.801762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5933a-eb57-4b56-a189-507b686e5180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:10.802449 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5933a-eb57-4b56-a189-507b686e5180 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:10.802746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5933a-eb57-4b56-a189-507b686e5180 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:10.803031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5933a-eb57-4b56-a189-507b686e5180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:10.803890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5933a-eb57-4b56-a189-507b686e5180 None admin] RBAC: Adding query filter params (role.id=0044294fcceb4b6ba0cb27567f2761d4, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user.id=f9bf4be1c5e94c578c2e02b627c30a56) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:10.804208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5933a-eb57-4b56-a189-507b686e5180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:10.805714 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5933a-eb57-4b56-a189-507b686e5180 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:10.813845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 50/97] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:10 2022] GET /identity/v3/role_assignments?user.id=f9bf4be1c5e94c578c2e02b627c30a56&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=0044294fcceb4b6ba0cb27567f2761d4 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:12.154881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc66da-749c-46c6-b86f-fb93701b4fb1 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:12.156033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc66da-749c-46c6-b86f-fb93701b4fb1 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:12.156549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc66da-749c-46c6-b86f-fb93701b4fb1 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:12.158396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 48/98] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:12.164230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba640c-db00-4863-81f7-3e724dc64e9b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:12.164729 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba640c-db00-4863-81f7-3e724dc64e9b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:12.164801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba640c-db00-4863-81f7-3e724dc64e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:12.194446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-81ba640c-db00-4863-81f7-3e724dc64e9b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:12.195685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-81ba640c-db00-4863-81f7-3e724dc64e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:12.273296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 51/99] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:12.296263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc66da-749c-46c6-b86f-fb93701b4fb1 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:12.356431 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d63b2-26fe-4be5-9336-53c0334de832 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:12.356753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d63b2-26fe-4be5-9336-53c0334de832 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:12.357009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d63b2-26fe-4be5-9336-53c0334de832 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:12.358473 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 52/100] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:34:12.366464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053aced-7bc4-4265-8fd7-2786f60c0be4 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:12.366880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053aced-7bc4-4265-8fd7-2786f60c0be4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:12.367277 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053aced-7bc4-4265-8fd7-2786f60c0be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:12.395152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-2053aced-7bc4-4265-8fd7-2786f60c0be4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:12.396378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2053aced-7bc4-4265-8fd7-2786f60c0be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:12.413859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:12.473016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc66da-749c-46c6-b86f-fb93701b4fb1 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:12.533003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 53/101] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:12.534741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac924943-943a-47ea-a7bc-2f3c44506c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:12.536025 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac924943-943a-47ea-a7bc-2f3c44506c4b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:12.536327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac924943-943a-47ea-a7bc-2f3c44506c4b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:12.536668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac924943-943a-47ea-a7bc-2f3c44506c4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:12.558891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac924943-943a-47ea-a7bc-2f3c44506c4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-ac924943-943a-47ea-a7bc-2f3c44506c4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:12.567409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:12.568230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:12.568983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 49/102] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:12.574913 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053aced-7bc4-4265-8fd7-2786f60c0be4 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:12.576169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac924943-943a-47ea-a7bc-2f3c44506c4b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:12.608888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:12.724522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac924943-943a-47ea-a7bc-2f3c44506c4b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:12.761616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e657e-1af5-41ae-8531-b9a9a5ea0f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:12.761616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e657e-1af5-41ae-8531-b9a9a5ea0f70 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:12.761616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e657e-1af5-41ae-8531-b9a9a5ea0f70 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:12.761963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e657e-1af5-41ae-8531-b9a9a5ea0f70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:12.764655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:12.775037 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e657e-1af5-41ae-8531-b9a9a5ea0f70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-802e657e-1af5-41ae-8531-b9a9a5ea0f70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:12.785077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:12.786756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:12.788047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 50/103] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:34:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:12.791953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053aced-7bc4-4265-8fd7-2786f60c0be4 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:12.793238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e657e-1af5-41ae-8531-b9a9a5ea0f70 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:12.867890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24605223-843b-46ec-8960-c534a7e40e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:12.868756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24605223-843b-46ec-8960-c534a7e40e85 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:12.869148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24605223-843b-46ec-8960-c534a7e40e85 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:12.869617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24605223-843b-46ec-8960-c534a7e40e85 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:12.870497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24605223-843b-46ec-8960-c534a7e40e85 None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:12.871219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24605223-843b-46ec-8960-c534a7e40e85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:12.872492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24605223-843b-46ec-8960-c534a7e40e85 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:12.893388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 54/104] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:12.912698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24605223-843b-46ec-8960-c534a7e40e85 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:12.924417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e657e-1af5-41ae-8531-b9a9a5ea0f70 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:12.971559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4331fb-43ef-4a73-ac6c-23516f1d694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:12.972551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4331fb-43ef-4a73-ac6c-23516f1d694f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:12.973089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4331fb-43ef-4a73-ac6c-23516f1d694f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:12.973620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4331fb-43ef-4a73-ac6c-23516f1d694f None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:12.974633 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4331fb-43ef-4a73-ac6c-23516f1d694f None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:12.975102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4331fb-43ef-4a73-ac6c-23516f1d694f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:12.977646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4331fb-43ef-4a73-ac6c-23516f1d694f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:12.996313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 51/105] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:13.002193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4331fb-43ef-4a73-ac6c-23516f1d694f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:13.034256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:13.061619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24605223-843b-46ec-8960-c534a7e40e85 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:13.082101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:13.111348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67178fc-4749-48ba-b5f8-ade44521e73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:13.112720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67178fc-4749-48ba-b5f8-ade44521e73c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:13.113211 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67178fc-4749-48ba-b5f8-ade44521e73c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:13.113720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67178fc-4749-48ba-b5f8-ade44521e73c None admin] PATH_INFO: `/v3/users/f9bf4be1c5e94c578c2e02b627c30a56` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:13.115198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67178fc-4749-48ba-b5f8-ade44521e73c None admin] RBAC: Authorizing `identity:get_user(user_id=f9bf4be1c5e94c578c2e02b627c30a56)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:13.123301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67178fc-4749-48ba-b5f8-ade44521e73c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:13.125548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 55/106] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:12 2022] GET /identity/v3/users/f9bf4be1c5e94c578c2e02b627c30a56 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:13.135973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4331fb-43ef-4a73-ac6c-23516f1d694f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:13.143846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67178fc-4749-48ba-b5f8-ade44521e73c None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:13.178841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4e766-2345-41e7-9a52-a89158625420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:13.179747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4e766-2345-41e7-9a52-a89158625420 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:13.180037 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4e766-2345-41e7-9a52-a89158625420 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:13.180319 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4e766-2345-41e7-9a52-a89158625420 None admin] PATH_INFO: `/v3/users/629381bc156b476a94122bdd0a64b60d` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:13.181587 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4e766-2345-41e7-9a52-a89158625420 None admin] RBAC: Authorizing `identity:get_user(user_id=629381bc156b476a94122bdd0a64b60d)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:13.183051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4e766-2345-41e7-9a52-a89158625420 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:13.184974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 52/107] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:13 2022] GET /identity/v3/users/629381bc156b476a94122bdd0a64b60d => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:13.190288 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4e766-2345-41e7-9a52-a89158625420 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:13.269735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67178fc-4749-48ba-b5f8-ade44521e73c None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:13.314172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9547802-3c18-407b-9951-b63ae40a4222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:13.315405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9547802-3c18-407b-9951-b63ae40a4222 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:13.315897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9547802-3c18-407b-9951-b63ae40a4222 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:13.316403 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9547802-3c18-407b-9951-b63ae40a4222 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:13.318674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9547802-3c18-407b-9951-b63ae40a4222 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:13.321039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9547802-3c18-407b-9951-b63ae40a4222 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:13.325689 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 56/108] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:13 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:13.334345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9547802-3c18-407b-9951-b63ae40a4222 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:13.368145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:13.422342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4e766-2345-41e7-9a52-a89158625420 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:13.463882 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:13.480580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1226c84-9001-40a1-b5ca-c52e22ea68b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:13.481477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1226c84-9001-40a1-b5ca-c52e22ea68b3 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:13.481769 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1226c84-9001-40a1-b5ca-c52e22ea68b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:13.482048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1226c84-9001-40a1-b5ca-c52e22ea68b3 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:13.487770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1226c84-9001-40a1-b5ca-c52e22ea68b3 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:13.493119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1226c84-9001-40a1-b5ca-c52e22ea68b3 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:13.497648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 53/109] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:13 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:13.500990 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1226c84-9001-40a1-b5ca-c52e22ea68b3 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:13.579715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9547802-3c18-407b-9951-b63ae40a4222 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:13.634812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f745a02-69e0-4bb3-8e75-f49a8816756c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:13.636425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f745a02-69e0-4bb3-8e75-f49a8816756c None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:13.637037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f745a02-69e0-4bb3-8e75-f49a8816756c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:13.637628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f745a02-69e0-4bb3-8e75-f49a8816756c None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/users/f9bf4be1c5e94c578c2e02b627c30a56/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:13.645396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f745a02-69e0-4bb3-8e75-f49a8816756c None admin] RBAC: Authorizing `identity:create_grant(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user_id=f9bf4be1c5e94c578c2e02b627c30a56, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:13.647578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f745a02-69e0-4bb3-8e75-f49a8816756c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:13.675178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 57/110] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:34:13 2022] PUT /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/users/f9bf4be1c5e94c578c2e02b627c30a56/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:13.683173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa27117-3d28-461c-bae9-a72d03ac0c08 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:13.683522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa27117-3d28-461c-bae9-a72d03ac0c08 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:13.683757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa27117-3d28-461c-bae9-a72d03ac0c08 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:13.685183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 58/111] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:13.691950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e737f5-4363-486b-a5eb-95de20447f6d None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:13.692252 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e737f5-4363-486b-a5eb-95de20447f6d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:13.692540 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e737f5-4363-486b-a5eb-95de20447f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:13.723661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1226c84-9001-40a1-b5ca-c52e22ea68b3 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:13.734053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-59e737f5-4363-486b-a5eb-95de20447f6d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:13.735677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-59e737f5-4363-486b-a5eb-95de20447f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:13.770703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab43f7-89f4-44a6-8da8-69989a2f429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:13.772641 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab43f7-89f4-44a6-8da8-69989a2f429a None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:13.772935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab43f7-89f4-44a6-8da8-69989a2f429a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:13.773228 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab43f7-89f4-44a6-8da8-69989a2f429a None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/users/629381bc156b476a94122bdd0a64b60d/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:13.778781 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab43f7-89f4-44a6-8da8-69989a2f429a None admin] RBAC: Authorizing `identity:create_grant(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user_id=629381bc156b476a94122bdd0a64b60d, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:13.780913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab43f7-89f4-44a6-8da8-69989a2f429a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:13.817746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 54/112] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:34:13 2022] PUT /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/users/629381bc156b476a94122bdd0a64b60d/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:13.900091 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 59/113] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:13.947912 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab43f7-89f4-44a6-8da8-69989a2f429a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:13.958939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690c340-2ade-4c7b-b1ee-7fa0b62e1ce7 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:13.959317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690c340-2ade-4c7b-b1ee-7fa0b62e1ce7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:13.959516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690c340-2ade-4c7b-b1ee-7fa0b62e1ce7 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:13.961186 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 60/114] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:34:13.972466 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec69dd-8ed5-4a37-973f-f1c188fef49a None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:13.972790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec69dd-8ed5-4a37-973f-f1c188fef49a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:13.973038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec69dd-8ed5-4a37-973f-f1c188fef49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:13.978735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:13.994579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-82ec69dd-8ed5-4a37-973f-f1c188fef49a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:13.996244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-82ec69dd-8ed5-4a37-973f-f1c188fef49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:14.031205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:14.099069 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 61/115] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:14.099991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab43f7-89f4-44a6-8da8-69989a2f429a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:14.105382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0e86d-1dac-4586-884f-8984f4332396 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:14.106324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0e86d-1dac-4586-884f-8984f4332396 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:14.106804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0e86d-1dac-4586-884f-8984f4332396 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:14.109773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 62/116] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:14.123049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c7c25-06cc-442d-8c99-4e6386c71ae4 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:14.124245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c7c25-06cc-442d-8c99-4e6386c71ae4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:14.124534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c7c25-06cc-442d-8c99-4e6386c71ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:14.157014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0e5c7c25-06cc-442d-8c99-4e6386c71ae4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:14.158296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0e5c7c25-06cc-442d-8c99-4e6386c71ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:14.166023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2dc16-8bb0-4a68-8160-090d9954bf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:14.166877 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2dc16-8bb0-4a68-8160-090d9954bf5b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:14.167171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2dc16-8bb0-4a68-8160-090d9954bf5b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:14.167571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2dc16-8bb0-4a68-8160-090d9954bf5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:14.181155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2dc16-8bb0-4a68-8160-090d9954bf5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-a1c2dc16-8bb0-4a68-8160-090d9954bf5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:14.190090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:14.191078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 55/117] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:14.192569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2dc16-8bb0-4a68-8160-090d9954bf5b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:14.246596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 63/118] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:14.247510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c7c25-06cc-442d-8c99-4e6386c71ae4 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:14.331588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:14.375668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2dc16-8bb0-4a68-8160-090d9954bf5b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:14.383272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c7c25-06cc-442d-8c99-4e6386c71ae4 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:14.392532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:14.415073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328edcba-c3a3-4360-b97f-f4b0ddf46869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:14.415797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328edcba-c3a3-4360-b97f-f4b0ddf46869 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:14.416090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328edcba-c3a3-4360-b97f-f4b0ddf46869 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:14.416377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328edcba-c3a3-4360-b97f-f4b0ddf46869 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:14.417629 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328edcba-c3a3-4360-b97f-f4b0ddf46869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:14.419097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328edcba-c3a3-4360-b97f-f4b0ddf46869 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:14.420903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 56/119] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:14.421714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328edcba-c3a3-4360-b97f-f4b0ddf46869 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:14.443240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d057fa8-bde8-4c21-9871-e5de729f01d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:14.444293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d057fa8-bde8-4c21-9871-e5de729f01d4 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:14.444778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d057fa8-bde8-4c21-9871-e5de729f01d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:14.445275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d057fa8-bde8-4c21-9871-e5de729f01d4 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:14.446247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d057fa8-bde8-4c21-9871-e5de729f01d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:14.446702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d057fa8-bde8-4c21-9871-e5de729f01d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:14.448426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d057fa8-bde8-4c21-9871-e5de729f01d4 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:14.463498 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 64/120] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:14.464169 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d057fa8-bde8-4c21-9871-e5de729f01d4 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:14.552502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328edcba-c3a3-4360-b97f-f4b0ddf46869 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:14.600232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c576259-0d28-4fdc-974e-1a00e4b8529f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:14.601993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c576259-0d28-4fdc-974e-1a00e4b8529f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:14.602527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c576259-0d28-4fdc-974e-1a00e4b8529f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:14.602992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c576259-0d28-4fdc-974e-1a00e4b8529f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:14.617488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c576259-0d28-4fdc-974e-1a00e4b8529f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:14.623659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d057fa8-bde8-4c21-9871-e5de729f01d4 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-1c576259-0d28-4fdc-974e-1a00e4b8529f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:14.628251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:14.629562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:14.630488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:14.637639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 57/121] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:14.637639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c576259-0d28-4fdc-974e-1a00e4b8529f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:14.654266 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:14.669087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb42b3c-a7d7-471a-a224-6d379e0940f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:14.669782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb42b3c-a7d7-471a-a224-6d379e0940f5 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:14.670110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb42b3c-a7d7-471a-a224-6d379e0940f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:14.670548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb42b3c-a7d7-471a-a224-6d379e0940f5 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:14.671490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb42b3c-a7d7-471a-a224-6d379e0940f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:14.673398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb42b3c-a7d7-471a-a224-6d379e0940f5 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:14.686289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:14.710668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 65/122] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:34:14 2022] POST /identity/v3/projects => generated 302 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:34:14.712375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3f631-b551-43dc-b461-210b08228127 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:14.712679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3f631-b551-43dc-b461-210b08228127 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:14.712971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3f631-b551-43dc-b461-210b08228127 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:14.714646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 66/123] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:14.715542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3f631-b551-43dc-b461-210b08228127 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:14.757583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c576259-0d28-4fdc-974e-1a00e4b8529f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:14.794479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40149f-bded-45c3-80ed-6bc31d1c1b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:14.795938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40149f-bded-45c3-80ed-6bc31d1c1b55 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:14.796225 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40149f-bded-45c3-80ed-6bc31d1c1b55 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:14.796503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40149f-bded-45c3-80ed-6bc31d1c1b55 None admin] PATH_INFO: `/v3/users/f9bf4be1c5e94c578c2e02b627c30a56` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:14.797835 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40149f-bded-45c3-80ed-6bc31d1c1b55 None admin] RBAC: Authorizing `identity:get_user(user_id=f9bf4be1c5e94c578c2e02b627c30a56)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:14.799302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40149f-bded-45c3-80ed-6bc31d1c1b55 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:14.801283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 58/124] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:14 2022] GET /identity/v3/users/f9bf4be1c5e94c578c2e02b627c30a56 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:14.802563 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636225d-c5a8-4fed-80f7-89f9a6b1d584 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:14.802836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636225d-c5a8-4fed-80f7-89f9a6b1d584 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:14.803117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636225d-c5a8-4fed-80f7-89f9a6b1d584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:14.846586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0636225d-c5a8-4fed-80f7-89f9a6b1d584 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:14.847843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0636225d-c5a8-4fed-80f7-89f9a6b1d584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:14.856526 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3f631-b551-43dc-b461-210b08228127 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:14.897496 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ebefe-5cc4-415d-b225-5070b12876ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:14.898301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ebefe-5cc4-415d-b225-5070b12876ae None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:14.898770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ebefe-5cc4-415d-b225-5070b12876ae None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:14.899261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ebefe-5cc4-415d-b225-5070b12876ae None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:14.900182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ebefe-5cc4-415d-b225-5070b12876ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:14.900501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ebefe-5cc4-415d-b225-5070b12876ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:14.902094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5ebefe-5cc4-415d-b225-5070b12876ae None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:14.912957 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 67/125] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:14.918310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ebefe-5cc4-415d-b225-5070b12876ae None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:15.006074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 59/126] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:15.006779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636225d-c5a8-4fed-80f7-89f9a6b1d584 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:15.021731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:15.068592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:15.124721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ebefe-5cc4-415d-b225-5070b12876ae None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:15.149014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636225d-c5a8-4fed-80f7-89f9a6b1d584 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:15.162968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024f1e5-f5e7-4c83-be02-2970afeeb2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:15.163846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f1e5-f5e7-4c83-be02-2970afeeb2d1 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:15.164137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f1e5-f5e7-4c83-be02-2970afeeb2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:15.164444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f1e5-f5e7-4c83-be02-2970afeeb2d1 None admin] PATH_INFO: `/v3/projects/88dbcaf6b71a49eeb0297a64781368f4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:15.165883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024f1e5-f5e7-4c83-be02-2970afeeb2d1 None admin] RBAC: Authorizing `identity:get_project(project_id=88dbcaf6b71a49eeb0297a64781368f4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:15.167354 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024f1e5-f5e7-4c83-be02-2970afeeb2d1 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:15.169488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 68/127] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:14 2022] GET /identity/v3/projects/88dbcaf6b71a49eeb0297a64781368f4 => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:15.170740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024f1e5-f5e7-4c83-be02-2970afeeb2d1 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:15.188299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7e339-2b97-4399-b1f6-3a1cc7c0137b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:15.189195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7e339-2b97-4399-b1f6-3a1cc7c0137b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:15.189513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7e339-2b97-4399-b1f6-3a1cc7c0137b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:15.189804 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7e339-2b97-4399-b1f6-3a1cc7c0137b None admin] PATH_INFO: `/v3/users/fa02e6ef015247398a8c719c2d90c7de` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:15.205042 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7e339-2b97-4399-b1f6-3a1cc7c0137b None admin] RBAC: Authorizing `identity:get_user(user_id=fa02e6ef015247398a8c719c2d90c7de)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:15.206838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7e339-2b97-4399-b1f6-3a1cc7c0137b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:15.208730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 60/128] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:15 2022] GET /identity/v3/users/fa02e6ef015247398a8c719c2d90c7de => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:15.209321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7e339-2b97-4399-b1f6-3a1cc7c0137b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:15.302641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024f1e5-f5e7-4c83-be02-2970afeeb2d1 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:15.316266 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:15.318999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:15.324107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7e339-2b97-4399-b1f6-3a1cc7c0137b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:15.358219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23e211-39f5-41b7-bc80-8ae8e8716dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:15.359153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23e211-39f5-41b7-bc80-8ae8e8716dc1 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:15.365387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23e211-39f5-41b7-bc80-8ae8e8716dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:15.365387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23e211-39f5-41b7-bc80-8ae8e8716dc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:15.365798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0d4c5-a266-406f-9a6b-b1999346a66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:15.365798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0d4c5-a266-406f-9a6b-b1999346a66f None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:15.365798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0d4c5-a266-406f-9a6b-b1999346a66f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:15.365798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0d4c5-a266-406f-9a6b-b1999346a66f None admin] PATH_INFO: `/v3/projects/88dbcaf6b71a49eeb0297a64781368f4/users/f9bf4be1c5e94c578c2e02b627c30a56/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:15.368061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0d4c5-a266-406f-9a6b-b1999346a66f None admin] RBAC: Authorizing `identity:create_grant(project_id=88dbcaf6b71a49eeb0297a64781368f4, user_id=f9bf4be1c5e94c578c2e02b627c30a56, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:15.369615 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0d4c5-a266-406f-9a6b-b1999346a66f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:15.383028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23e211-39f5-41b7-bc80-8ae8e8716dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-ad23e211-39f5-41b7-bc80-8ae8e8716dc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:15.391979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:15.392960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:15.393795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 69/129] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:15.395777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23e211-39f5-41b7-bc80-8ae8e8716dc1 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:15.395878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 61/130] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:34:15 2022] PUT /identity/v3/projects/88dbcaf6b71a49eeb0297a64781368f4/users/f9bf4be1c5e94c578c2e02b627c30a56/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:15.402190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0d4c5-a266-406f-9a6b-b1999346a66f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:15.583725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0d4c5-a266-406f-9a6b-b1999346a66f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:15.625750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f07d5d-5ebe-4074-ab25-de9f3cf215ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:15.626448 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f07d5d-5ebe-4074-ab25-de9f3cf215ef None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:15.626740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f07d5d-5ebe-4074-ab25-de9f3cf215ef None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:15.627233 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f07d5d-5ebe-4074-ab25-de9f3cf215ef None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:15.628026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f07d5d-5ebe-4074-ab25-de9f3cf215ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:15.628344 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f07d5d-5ebe-4074-ab25-de9f3cf215ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:15.629926 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f07d5d-5ebe-4074-ab25-de9f3cf215ef None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:15.640518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 62/131] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:15.646799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f07d5d-5ebe-4074-ab25-de9f3cf215ef None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:15.671233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23e211-39f5-41b7-bc80-8ae8e8716dc1 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:15.679289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:15.713683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f083b4-ad85-4764-a7fe-41ed045bd2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:15.715349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f083b4-ad85-4764-a7fe-41ed045bd2d0 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:15.715853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f083b4-ad85-4764-a7fe-41ed045bd2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:15.716310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f083b4-ad85-4764-a7fe-41ed045bd2d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:15.797934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:15.811555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f07d5d-5ebe-4074-ab25-de9f3cf215ef None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:15.828361 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f083b4-ad85-4764-a7fe-41ed045bd2d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:15.830037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f083b4-ad85-4764-a7fe-41ed045bd2d0 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:15.895020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba32222-c07b-492d-8bf4-670d115c52ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:15.895842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba32222-c07b-492d-8bf4-670d115c52ad None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:15.896127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba32222-c07b-492d-8bf4-670d115c52ad None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:15.896403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba32222-c07b-492d-8bf4-670d115c52ad None admin] PATH_INFO: `/v3/users/f9bf4be1c5e94c578c2e02b627c30a56` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:15.902137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba32222-c07b-492d-8bf4-670d115c52ad None admin] RBAC: Authorizing `identity:get_user(user_id=f9bf4be1c5e94c578c2e02b627c30a56)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:15.903631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba32222-c07b-492d-8bf4-670d115c52ad None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:15.905415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 63/132] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:15 2022] GET /identity/v3/users/f9bf4be1c5e94c578c2e02b627c30a56 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:15.927540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba32222-c07b-492d-8bf4-670d115c52ad None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:15.943051 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 70/133] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:15.948898 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f083b4-ad85-4764-a7fe-41ed045bd2d0 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:16.028260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:16.065852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba32222-c07b-492d-8bf4-670d115c52ad None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:16.081759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:16.085764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f083b4-ad85-4764-a7fe-41ed045bd2d0 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:16.135190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e268406-d3f2-4271-bba1-79c4c288b972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:16.136082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e268406-d3f2-4271-bba1-79c4c288b972 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:16.136366 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e268406-d3f2-4271-bba1-79c4c288b972 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:16.136647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e268406-d3f2-4271-bba1-79c4c288b972 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:16.142250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e268406-d3f2-4271-bba1-79c4c288b972 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:16.144000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e268406-d3f2-4271-bba1-79c4c288b972 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:16.146965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 64/134] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:15 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:16.152696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e268406-d3f2-4271-bba1-79c4c288b972 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:16.157988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cd9d7-6266-4be2-86ff-de57f0ce3d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:16.158886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd9d7-6266-4be2-86ff-de57f0ce3d66 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:16.159178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd9d7-6266-4be2-86ff-de57f0ce3d66 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:16.159463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd9d7-6266-4be2-86ff-de57f0ce3d66 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:16.160845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cd9d7-6266-4be2-86ff-de57f0ce3d66 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:16.166753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cd9d7-6266-4be2-86ff-de57f0ce3d66 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:16.168862 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 71/135] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:15 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:16.175260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cd9d7-6266-4be2-86ff-de57f0ce3d66 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:16.301399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e268406-d3f2-4271-bba1-79c4c288b972 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:16.346120 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12797318-3af5-42b1-bb5f-8fbc2d7e5633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:16.347048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12797318-3af5-42b1-bb5f-8fbc2d7e5633 None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:16.347499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12797318-3af5-42b1-bb5f-8fbc2d7e5633 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:16.347919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12797318-3af5-42b1-bb5f-8fbc2d7e5633 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/users/f9bf4be1c5e94c578c2e02b627c30a56/roles/0044294fcceb4b6ba0cb27567f2761d4` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:16.352605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12797318-3af5-42b1-bb5f-8fbc2d7e5633 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user_id=f9bf4be1c5e94c578c2e02b627c30a56, role_id=0044294fcceb4b6ba0cb27567f2761d4)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:16.354111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12797318-3af5-42b1-bb5f-8fbc2d7e5633 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:16.388720 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 65/136] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:34:16 2022] PUT /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/users/f9bf4be1c5e94c578c2e02b627c30a56/roles/0044294fcceb4b6ba0cb27567f2761d4 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:34:16.396776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cd9d7-6266-4be2-86ff-de57f0ce3d66 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:16.458159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8abc4-2bf9-4e11-9a26-b724ef529ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:16.458750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8abc4-2bf9-4e11-9a26-b724ef529ae1 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:16.459039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8abc4-2bf9-4e11-9a26-b724ef529ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:16.459327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8abc4-2bf9-4e11-9a26-b724ef529ae1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:16.460188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8abc4-2bf9-4e11-9a26-b724ef529ae1 None admin] RBAC: Adding query filter params (role.id=74ea4fc9cad8496a831bb325feb84d4c, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user.id=fa02e6ef015247398a8c719c2d90c7de) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:16.460505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8abc4-2bf9-4e11-9a26-b724ef529ae1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:16.462024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8abc4-2bf9-4e11-9a26-b724ef529ae1 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:16.464099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:16.474787 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 72/137] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:16 2022] GET /identity/v3/role_assignments?user.id=fa02e6ef015247398a8c719c2d90c7de&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=74ea4fc9cad8496a831bb325feb84d4c => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:16.750293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5e5b4-59e4-4d25-b804-da4c8c746dbf None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:16.750622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5e5b4-59e4-4d25-b804-da4c8c746dbf None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:16.750803 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5e5b4-59e4-4d25-b804-da4c8c746dbf None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:16.752157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 66/138] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:16.758940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6361de-7c51-4555-ae30-bd43a225a8ed None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:16.759232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6361de-7c51-4555-ae30-bd43a225a8ed None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:16.759511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6361de-7c51-4555-ae30-bd43a225a8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:16.780081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-de6361de-7c51-4555-ae30-bd43a225a8ed None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:16.781274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-de6361de-7c51-4555-ae30-bd43a225a8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:16.871657 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 73/139] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:16.907253 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5e5b4-59e4-4d25-b804-da4c8c746dbf None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:16.911689 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01920628-9309-478d-a34c-008e712b95ff None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:16.911963 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01920628-9309-478d-a34c-008e712b95ff None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:16.912245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01920628-9309-478d-a34c-008e712b95ff None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:16.913928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 74/140] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:16.921385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cd341-7e89-44e6-87f7-1c1147f5269f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:16.921385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cd341-7e89-44e6-87f7-1c1147f5269f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:16.921385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cd341-7e89-44e6-87f7-1c1147f5269f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:16.932947 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:16.954631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-686cd341-7e89-44e6-87f7-1c1147f5269f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:16.955885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-686cd341-7e89-44e6-87f7-1c1147f5269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:17.076883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5e5b4-59e4-4d25-b804-da4c8c746dbf None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:17.095386 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 75/141] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:17.125786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc6c4d-af31-47c0-8e21-2f1cc1dbfcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:17.126977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc6c4d-af31-47c0-8e21-2f1cc1dbfcf9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:17.127462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc6c4d-af31-47c0-8e21-2f1cc1dbfcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:17.127936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc6c4d-af31-47c0-8e21-2f1cc1dbfcf9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:17.134461 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cd341-7e89-44e6-87f7-1c1147f5269f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:17.148511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc6c4d-af31-47c0-8e21-2f1cc1dbfcf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-adfc6c4d-af31-47c0-8e21-2f1cc1dbfcf9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:17.165815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:17.166735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:17.167496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:17.172783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 67/142] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:17.177030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc6c4d-af31-47c0-8e21-2f1cc1dbfcf9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:17.235077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:17.307962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cd341-7e89-44e6-87f7-1c1147f5269f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:17.317703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:17.381956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc6c4d-af31-47c0-8e21-2f1cc1dbfcf9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:17.395943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8680ca2-b653-433f-927f-1ea398086000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:17.396747 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8680ca2-b653-433f-927f-1ea398086000 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:17.397028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8680ca2-b653-433f-927f-1ea398086000 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:17.397604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8680ca2-b653-433f-927f-1ea398086000 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:17.442034 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8680ca2-b653-433f-927f-1ea398086000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-e8680ca2-b653-433f-927f-1ea398086000 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:17.450142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:17.451015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:17.451800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 76/143] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:34:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:17.457189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8680ca2-b653-433f-927f-1ea398086000 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:17.498677 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc664cd7-cd3f-4d65-9c95-f4cfbe81fca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:17.499368 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc664cd7-cd3f-4d65-9c95-f4cfbe81fca3 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:17.499655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc664cd7-cd3f-4d65-9c95-f4cfbe81fca3 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:17.499936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc664cd7-cd3f-4d65-9c95-f4cfbe81fca3 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:17.500705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc664cd7-cd3f-4d65-9c95-f4cfbe81fca3 None admin] RBAC: Adding query filter params (name=member) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:17.501016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc664cd7-cd3f-4d65-9c95-f4cfbe81fca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:17.502433 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc664cd7-cd3f-4d65-9c95-f4cfbe81fca3 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:17.519971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 68/144] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:17.525237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc664cd7-cd3f-4d65-9c95-f4cfbe81fca3 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:17.697470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8680ca2-b653-433f-927f-1ea398086000 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:17.730996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:17.749612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc664cd7-cd3f-4d65-9c95-f4cfbe81fca3 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:17.771118 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf9465-624d-4b42-9e2c-f9b9d67f728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:17.771826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9465-624d-4b42-9e2c-f9b9d67f728b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:17.772117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9465-624d-4b42-9e2c-f9b9d67f728b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:17.772403 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9465-624d-4b42-9e2c-f9b9d67f728b None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:17.773181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf9465-624d-4b42-9e2c-f9b9d67f728b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:17.773521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf9465-624d-4b42-9e2c-f9b9d67f728b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:17.774934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf9465-624d-4b42-9e2c-f9b9d67f728b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:17.799397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 77/145] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:17.822708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf9465-624d-4b42-9e2c-f9b9d67f728b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:17.823018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c02cf-2193-49d9-844b-024d5a76f098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:17.824269 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c02cf-2193-49d9-844b-024d5a76f098 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:17.824557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c02cf-2193-49d9-844b-024d5a76f098 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:17.824837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c02cf-2193-49d9-844b-024d5a76f098 None admin] PATH_INFO: `/v3/users/f9bf4be1c5e94c578c2e02b627c30a56` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:17.826078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c02cf-2193-49d9-844b-024d5a76f098 None admin] RBAC: Authorizing `identity:get_user(user_id=f9bf4be1c5e94c578c2e02b627c30a56)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:17.827577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c02cf-2193-49d9-844b-024d5a76f098 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:17.829485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 69/146] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:17 2022] GET /identity/v3/users/f9bf4be1c5e94c578c2e02b627c30a56 => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:17.834933 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c02cf-2193-49d9-844b-024d5a76f098 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:17.847438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:17.987135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c02cf-2193-49d9-844b-024d5a76f098 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:18.046806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f901540-e0d4-4a8c-b360-3fbdf46c5ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:18.047481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f901540-e0d4-4a8c-b360-3fbdf46c5ee8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:18.047911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f901540-e0d4-4a8c-b360-3fbdf46c5ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:18.048192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f901540-e0d4-4a8c-b360-3fbdf46c5ee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:18.132918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf9465-624d-4b42-9e2c-f9b9d67f728b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:18.165090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f901540-e0d4-4a8c-b360-3fbdf46c5ee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:18.174948 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f901540-e0d4-4a8c-b360-3fbdf46c5ee8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:18.183790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:18.204748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad3e0c-f446-4d6b-aada-5dd6e3bbf85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:18.205677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad3e0c-f446-4d6b-aada-5dd6e3bbf85a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:18.205965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad3e0c-f446-4d6b-aada-5dd6e3bbf85a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:18.206239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad3e0c-f446-4d6b-aada-5dd6e3bbf85a None admin] PATH_INFO: `/v3/users/629381bc156b476a94122bdd0a64b60d` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:18.207582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad3e0c-f446-4d6b-aada-5dd6e3bbf85a None admin] RBAC: Authorizing `identity:get_user(user_id=629381bc156b476a94122bdd0a64b60d)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:18.209025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad3e0c-f446-4d6b-aada-5dd6e3bbf85a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:18.211223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 78/147] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:17 2022] GET /identity/v3/users/629381bc156b476a94122bdd0a64b60d => generated 247 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:18.235406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad3e0c-f446-4d6b-aada-5dd6e3bbf85a None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:18.313569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 70/148] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:34:18.318691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f901540-e0d4-4a8c-b360-3fbdf46c5ee8 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:18.333752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:18.389887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad3e0c-f446-4d6b-aada-5dd6e3bbf85a None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:18.431138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4d137-7825-454a-b562-cc49e7968d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:18.431920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4d137-7825-454a-b562-cc49e7968d09 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:18.432084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4d137-7825-454a-b562-cc49e7968d09 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:18.432372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4d137-7825-454a-b562-cc49e7968d09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:18.509583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f901540-e0d4-4a8c-b360-3fbdf46c5ee8 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:18.518668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:18.553306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa65bac-2729-4fed-96c2-2ddffa45e24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:18.554237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa65bac-2729-4fed-96c2-2ddffa45e24d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:18.554523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa65bac-2729-4fed-96c2-2ddffa45e24d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:18.554801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa65bac-2729-4fed-96c2-2ddffa45e24d None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:18.556912 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa65bac-2729-4fed-96c2-2ddffa45e24d None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:18.558731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa65bac-2729-4fed-96c2-2ddffa45e24d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:18.561402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 71/149] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:18 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:18.562791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4498b-9282-484e-b127-d99023123123 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:18.563085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4498b-9282-484e-b127-d99023123123 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:18.563493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4498b-9282-484e-b127-d99023123123 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:18.565009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 72/150] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:18.566644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bab57-5712-40c5-91b8-55b3ebbc99db None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:18.566942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bab57-5712-40c5-91b8-55b3ebbc99db None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:18.567224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bab57-5712-40c5-91b8-55b3ebbc99db None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:18.568660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 73/151] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:18.569538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bab57-5712-40c5-91b8-55b3ebbc99db None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:18.579942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4d137-7825-454a-b562-cc49e7968d09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:18.581783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4d137-7825-454a-b562-cc49e7968d09 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:18.642965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:18.724018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bab57-5712-40c5-91b8-55b3ebbc99db None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:18.727297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 79/152] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:18.728845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d00b7-9bc4-45d3-b342-145275fb884b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:18.729451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d00b7-9bc4-45d3-b342-145275fb884b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:18.729929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d00b7-9bc4-45d3-b342-145275fb884b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:18.750812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-b93d00b7-9bc4-45d3-b342-145275fb884b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:18.752464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b93d00b7-9bc4-45d3-b342-145275fb884b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:18.770077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23985c-c363-44ed-847a-b557bb6a9d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:18.770818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23985c-c363-44ed-847a-b557bb6a9d16 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:18.771119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23985c-c363-44ed-847a-b557bb6a9d16 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:18.771401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23985c-c363-44ed-847a-b557bb6a9d16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:18.772566 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23985c-c363-44ed-847a-b557bb6a9d16 None admin] RBAC: Adding query filter params (role.id=d94de411ec0a40be92c04950e6edcac7, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user.id=f9bf4be1c5e94c578c2e02b627c30a56) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:18.772928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23985c-c363-44ed-847a-b557bb6a9d16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:18.774693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23985c-c363-44ed-847a-b557bb6a9d16 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:18.789479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 74/153] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:18 2022] GET /identity/v3/role_assignments?user.id=f9bf4be1c5e94c578c2e02b627c30a56&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:18.791114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53e810-362e-426b-8982-46cffbfa9329 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:18.791411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53e810-362e-426b-8982-46cffbfa9329 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:18.791688 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53e810-362e-426b-8982-46cffbfa9329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:18.829490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-3a53e810-362e-426b-8982-46cffbfa9329 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:18.831118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3a53e810-362e-426b-8982-46cffbfa9329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:18.889777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:18.892706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 80/154] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:18.894242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d00b7-9bc4-45d3-b342-145275fb884b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:19.016035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 75/155] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:19.017263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53e810-362e-426b-8982-46cffbfa9329 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:19.036227 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:19.054710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d00b7-9bc4-45d3-b342-145275fb884b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:19.106114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9537a-57f3-40db-b8a6-d86d53be1635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:19.106914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9537a-57f3-40db-b8a6-d86d53be1635 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.107194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9537a-57f3-40db-b8a6-d86d53be1635 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.108025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9537a-57f3-40db-b8a6-d86d53be1635 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.109343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf9537a-57f3-40db-b8a6-d86d53be1635 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:19.110841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf9537a-57f3-40db-b8a6-d86d53be1635 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:19.112958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 81/156] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:18 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:19.113807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9537a-57f3-40db-b8a6-d86d53be1635 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:19.149108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53e810-362e-426b-8982-46cffbfa9329 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:19.194585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d803b01-4624-4cd3-ab84-004c59f1b79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:19.195399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d803b01-4624-4cd3-ab84-004c59f1b79b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.195680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d803b01-4624-4cd3-ab84-004c59f1b79b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.195953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d803b01-4624-4cd3-ab84-004c59f1b79b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.208022 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d803b01-4624-4cd3-ab84-004c59f1b79b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-5d803b01-4624-4cd3-ab84-004c59f1b79b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:19.216626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:19.218425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 76/157] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:19.219903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d803b01-4624-4cd3-ab84-004c59f1b79b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:19.231880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9537a-57f3-40db-b8a6-d86d53be1635 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:19.261408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:19.284553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add832c-712e-4c8b-aa72-03877e0d631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:19.285390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add832c-712e-4c8b-aa72-03877e0d631a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.285687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add832c-712e-4c8b-aa72-03877e0d631a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.285964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add832c-712e-4c8b-aa72-03877e0d631a None admin] PATH_INFO: `/v3/domains/default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.288368 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add832c-712e-4c8b-aa72-03877e0d631a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:19.290940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add832c-712e-4c8b-aa72-03877e0d631a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:19.292980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 82/158] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:19.293888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add832c-712e-4c8b-aa72-03877e0d631a None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:19.336654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:19.376761 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d803b01-4624-4cd3-ab84-004c59f1b79b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:19.430776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0cd28-76fa-4e00-9b92-4ea85004ee88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:19.431815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0cd28-76fa-4e00-9b92-4ea85004ee88 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.432381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0cd28-76fa-4e00-9b92-4ea85004ee88 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.432670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0cd28-76fa-4e00-9b92-4ea85004ee88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.433570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0cd28-76fa-4e00-9b92-4ea85004ee88 None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user.id=629381bc156b476a94122bdd0a64b60d) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:19.433891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0cd28-76fa-4e00-9b92-4ea85004ee88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:19.436453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0cd28-76fa-4e00-9b92-4ea85004ee88 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:19.441706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add832c-712e-4c8b-aa72-03877e0d631a None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:19.443250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 77/159] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:19 2022] GET /identity/v3/role_assignments?user.id=629381bc156b476a94122bdd0a64b60d&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=85420c68300642aabfd730fb8fef8347 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:19.443834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0cd28-76fa-4e00-9b92-4ea85004ee88 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:19.493850 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395715a-b9ec-4f75-b80f-acd495b19662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:19.494945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395715a-b9ec-4f75-b80f-acd495b19662 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.495541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395715a-b9ec-4f75-b80f-acd495b19662 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.496025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395715a-b9ec-4f75-b80f-acd495b19662 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.497155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395715a-b9ec-4f75-b80f-acd495b19662 None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:19.497709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395715a-b9ec-4f75-b80f-acd495b19662 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:19.499181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395715a-b9ec-4f75-b80f-acd495b19662 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:19.509667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 83/160] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:19.516537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490939e-df90-444a-acff-db8ab4a3abd6 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.517219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490939e-df90-444a-acff-db8ab4a3abd6 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.517535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490939e-df90-444a-acff-db8ab4a3abd6 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.519222 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 84/161] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:19.520297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768441d-814c-4b6a-9efc-819b8820f33e None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.520587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768441d-814c-4b6a-9efc-819b8820f33e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.520868 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768441d-814c-4b6a-9efc-819b8820f33e None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.522368 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 85/162] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:19.522978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768441d-814c-4b6a-9efc-819b8820f33e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:19.577068 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:19.588617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0cd28-76fa-4e00-9b92-4ea85004ee88 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:19.613185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:19.635276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba088e-7350-47de-ad74-82aacd3469fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:19.635882 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba088e-7350-47de-ad74-82aacd3469fb None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.636220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba088e-7350-47de-ad74-82aacd3469fb None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.636778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba088e-7350-47de-ad74-82aacd3469fb None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.638071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba088e-7350-47de-ad74-82aacd3469fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:19.639413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba088e-7350-47de-ad74-82aacd3469fb None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:19.641334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768441d-814c-4b6a-9efc-819b8820f33e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:19.667135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 78/163] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:34:19 2022] POST /identity/v3/users => generated 283 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:34:19.668416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd821e7-7ec7-49dc-bf32-3d66a6270cd7 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.668800 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd821e7-7ec7-49dc-bf32-3d66a6270cd7 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.669091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd821e7-7ec7-49dc-bf32-3d66a6270cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.684538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be51fd7-1ccb-4656-b32f-53e7e270d69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:19.686033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be51fd7-1ccb-4656-b32f-53e7e270d69a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.686330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be51fd7-1ccb-4656-b32f-53e7e270d69a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.686611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be51fd7-1ccb-4656-b32f-53e7e270d69a None admin] PATH_INFO: `/v3/users/f9bf4be1c5e94c578c2e02b627c30a56` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.689327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be51fd7-1ccb-4656-b32f-53e7e270d69a None admin] RBAC: Authorizing `identity:get_user(user_id=f9bf4be1c5e94c578c2e02b627c30a56)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:19.690395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-9dd821e7-7ec7-49dc-bf32-3d66a6270cd7 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:19.690918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be51fd7-1ccb-4656-b32f-53e7e270d69a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:19.692100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-9dd821e7-7ec7-49dc-bf32-3d66a6270cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:19.692699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 86/164] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:19 2022] GET /identity/v3/users/f9bf4be1c5e94c578c2e02b627c30a56 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:19.699766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400a068-757f-44bd-8de6-dee1ca8fe8ab None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.699766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400a068-757f-44bd-8de6-dee1ca8fe8ab None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.699766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400a068-757f-44bd-8de6-dee1ca8fe8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.718058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a400a068-757f-44bd-8de6-dee1ca8fe8ab None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:19.719732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a400a068-757f-44bd-8de6-dee1ca8fe8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:19.788917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 79/165] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:19.794373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd821e7-7ec7-49dc-bf32-3d66a6270cd7 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:19.796465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 87/166] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:19.817720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400a068-757f-44bd-8de6-dee1ca8fe8ab None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:19.895111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:19.916317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:19.927494 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd821e7-7ec7-49dc-bf32-3d66a6270cd7 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:19.948369 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400a068-757f-44bd-8de6-dee1ca8fe8ab None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:19.988219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538de96-1a6b-447e-b39d-32c9e2c0c722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:19.988984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538de96-1a6b-447e-b39d-32c9e2c0c722 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.989272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538de96-1a6b-447e-b39d-32c9e2c0c722 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.989579 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538de96-1a6b-447e-b39d-32c9e2c0c722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:19.991857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf937260-d34c-45d7-9a77-b83dd0876452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:19.992666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf937260-d34c-45d7-9a77-b83dd0876452 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:19.992959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf937260-d34c-45d7-9a77-b83dd0876452 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:19.993243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf937260-d34c-45d7-9a77-b83dd0876452 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:20.004578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf937260-d34c-45d7-9a77-b83dd0876452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-bf937260-d34c-45d7-9a77-b83dd0876452 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:20.014872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:20.015839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:20.016756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 88/167] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:34:20.017425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf937260-d34c-45d7-9a77-b83dd0876452 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:20.079328 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538de96-1a6b-447e-b39d-32c9e2c0c722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:20.081773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538de96-1a6b-447e-b39d-32c9e2c0c722 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:20.162293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf937260-d34c-45d7-9a77-b83dd0876452 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:20.193570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:20.206938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46f7c0-7933-47ff-99de-09cf3fb4f33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:20.207994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f7c0-7933-47ff-99de-09cf3fb4f33f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:20.208475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f7c0-7933-47ff-99de-09cf3fb4f33f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:20.208984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f7c0-7933-47ff-99de-09cf3fb4f33f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:20.212565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:20.225655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46f7c0-7933-47ff-99de-09cf3fb4f33f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-4e46f7c0-7933-47ff-99de-09cf3fb4f33f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:20.234588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:20.236098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 89/168] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:20.237326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46f7c0-7933-47ff-99de-09cf3fb4f33f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:20.250134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 80/169] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:20.252563 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538de96-1a6b-447e-b39d-32c9e2c0c722 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:20.405522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538de96-1a6b-447e-b39d-32c9e2c0c722 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:20.425190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46f7c0-7933-47ff-99de-09cf3fb4f33f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:20.457925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f08863-7d9d-41de-baa3-26d088818ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:20.458910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f08863-7d9d-41de-baa3-26d088818ded None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:20.459055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f08863-7d9d-41de-baa3-26d088818ded None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:20.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f08863-7d9d-41de-baa3-26d088818ded None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:20.460109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f08863-7d9d-41de-baa3-26d088818ded None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:20.460462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f08863-7d9d-41de-baa3-26d088818ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:20.461945 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f08863-7d9d-41de-baa3-26d088818ded None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:20.471872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 81/170] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:20.472889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f08863-7d9d-41de-baa3-26d088818ded None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:20.478094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90f64e-3aa7-4cdd-a9fd-07aa5cd4425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:20.479382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90f64e-3aa7-4cdd-a9fd-07aa5cd4425f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:20.479834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90f64e-3aa7-4cdd-a9fd-07aa5cd4425f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:20.480253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90f64e-3aa7-4cdd-a9fd-07aa5cd4425f None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:20.481176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90f64e-3aa7-4cdd-a9fd-07aa5cd4425f None admin] RBAC: Adding query filter params (name=reader) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:20.481671 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90f64e-3aa7-4cdd-a9fd-07aa5cd4425f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:20.483774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90f64e-3aa7-4cdd-a9fd-07aa5cd4425f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:20.499679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 90/171] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:20.500698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90f64e-3aa7-4cdd-a9fd-07aa5cd4425f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:20.533779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:20.605497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:20.657740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f08863-7d9d-41de-baa3-26d088818ded None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:20.693461 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90f64e-3aa7-4cdd-a9fd-07aa5cd4425f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:20.738574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8906a7-7e03-4916-8c8c-f96d76b96901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:20.740390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8906a7-7e03-4916-8c8c-f96d76b96901 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:20.740951 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8906a7-7e03-4916-8c8c-f96d76b96901 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:20.743628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8906a7-7e03-4916-8c8c-f96d76b96901 None admin] PATH_INFO: `/v3/users/f9bf4be1c5e94c578c2e02b627c30a56` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:20.745697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28423cb-02d5-41c0-a713-5c3266ac5a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:20.746453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8906a7-7e03-4916-8c8c-f96d76b96901 None admin] RBAC: Authorizing `identity:get_user(user_id=f9bf4be1c5e94c578c2e02b627c30a56)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:20.747211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28423cb-02d5-41c0-a713-5c3266ac5a22 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:20.747870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28423cb-02d5-41c0-a713-5c3266ac5a22 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:20.748375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28423cb-02d5-41c0-a713-5c3266ac5a22 None admin] PATH_INFO: `/v3/projects/88dbcaf6b71a49eeb0297a64781368f4` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:20.748812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8906a7-7e03-4916-8c8c-f96d76b96901 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:20.750491 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28423cb-02d5-41c0-a713-5c3266ac5a22 None admin] RBAC: Authorizing `identity:get_project(project_id=88dbcaf6b71a49eeb0297a64781368f4)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:20.757394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 91/172] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:20 2022] GET /identity/v3/users/f9bf4be1c5e94c578c2e02b627c30a56 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:20.757573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28423cb-02d5-41c0-a713-5c3266ac5a22 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:20.757573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 82/173] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:20 2022] GET /identity/v3/projects/88dbcaf6b71a49eeb0297a64781368f4 => generated 312 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:20.758212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28423cb-02d5-41c0-a713-5c3266ac5a22 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:20.758989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8906a7-7e03-4916-8c8c-f96d76b96901 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:20.958791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28423cb-02d5-41c0-a713-5c3266ac5a22 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:20.973732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:20.983688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8906a7-7e03-4916-8c8c-f96d76b96901 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:21.031253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:21.036232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fc89c-9b1d-402a-841c-65a2bf08ee47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:21.037386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc89c-9b1d-402a-841c-65a2bf08ee47 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:21.039196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc89c-9b1d-402a-841c-65a2bf08ee47 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:21.039536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc89c-9b1d-402a-841c-65a2bf08ee47 None admin] PATH_INFO: `/v3/users/fa02e6ef015247398a8c719c2d90c7de` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:21.041996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8fc89c-9b1d-402a-841c-65a2bf08ee47 None admin] RBAC: Authorizing `identity:get_user(user_id=fa02e6ef015247398a8c719c2d90c7de)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:21.043511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8fc89c-9b1d-402a-841c-65a2bf08ee47 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:21.045851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 83/174] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:20 2022] GET /identity/v3/users/fa02e6ef015247398a8c719c2d90c7de => generated 289 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:21.046659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fc89c-9b1d-402a-841c-65a2bf08ee47 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:21.057508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d173c3-2d6f-4d75-9b46-f5bd49f3df13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:21.058176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d173c3-2d6f-4d75-9b46-f5bd49f3df13 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:21.058464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d173c3-2d6f-4d75-9b46-f5bd49f3df13 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:21.058745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d173c3-2d6f-4d75-9b46-f5bd49f3df13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:21.180471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fc89c-9b1d-402a-841c-65a2bf08ee47 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:21.228383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f0e5a-be17-4ebf-ba24-10e12b4213fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:21.228951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f0e5a-be17-4ebf-ba24-10e12b4213fc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:21.229242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f0e5a-be17-4ebf-ba24-10e12b4213fc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:21.229753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f0e5a-be17-4ebf-ba24-10e12b4213fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:21.230612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f0e5a-be17-4ebf-ba24-10e12b4213fc None admin] RBAC: Adding query filter params (role.id=d94de411ec0a40be92c04950e6edcac7, scope.project.id=88dbcaf6b71a49eeb0297a64781368f4, user.id=f9bf4be1c5e94c578c2e02b627c30a56) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:21.230930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f0e5a-be17-4ebf-ba24-10e12b4213fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:21.235462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f0e5a-be17-4ebf-ba24-10e12b4213fc None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:21.242839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d173c3-2d6f-4d75-9b46-f5bd49f3df13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:21.244308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d173c3-2d6f-4d75-9b46-f5bd49f3df13 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:21.248692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 84/175] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:21 2022] GET /identity/v3/role_assignments?user.id=f9bf4be1c5e94c578c2e02b627c30a56&scope.project.id=88dbcaf6b71a49eeb0297a64781368f4&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:21.249561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f0e5a-be17-4ebf-ba24-10e12b4213fc None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:21.373832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:21.376572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 92/176] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:21.385410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d173c3-2d6f-4d75-9b46-f5bd49f3df13 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:21.426400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:21.538332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d173c3-2d6f-4d75-9b46-f5bd49f3df13 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:21.577237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143226f-4483-4633-92e9-c094480b1fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:21.578148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143226f-4483-4633-92e9-c094480b1fed None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:21.578434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143226f-4483-4633-92e9-c094480b1fed None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:21.578715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143226f-4483-4633-92e9-c094480b1fed None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:21.580176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143226f-4483-4633-92e9-c094480b1fed None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:21.581698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143226f-4483-4633-92e9-c094480b1fed None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:21.584247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 93/177] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:21 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:21.590887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143226f-4483-4633-92e9-c094480b1fed None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:21.593186 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f0e5a-be17-4ebf-ba24-10e12b4213fc None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:21.670640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85c1f5-82e2-4827-a16f-dd03ff6bd9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:21.672176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85c1f5-82e2-4827-a16f-dd03ff6bd9f1 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:21.672636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85c1f5-82e2-4827-a16f-dd03ff6bd9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:21.673079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85c1f5-82e2-4827-a16f-dd03ff6bd9f1 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:21.675180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85c1f5-82e2-4827-a16f-dd03ff6bd9f1 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:21.677715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85c1f5-82e2-4827-a16f-dd03ff6bd9f1 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:21.682001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 85/178] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:21 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:21.687821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85c1f5-82e2-4827-a16f-dd03ff6bd9f1 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:21.745752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:21.771751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143226f-4483-4633-92e9-c094480b1fed None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:21.858100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85c1f5-82e2-4827-a16f-dd03ff6bd9f1 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:21.873746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:21.882120 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84bcd2-287f-430a-8635-2e4e5c789941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:21.882814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84bcd2-287f-430a-8635-2e4e5c789941 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:21.883105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84bcd2-287f-430a-8635-2e4e5c789941 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:21.883389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84bcd2-287f-430a-8635-2e4e5c789941 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:21.884261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84bcd2-287f-430a-8635-2e4e5c789941 None admin] RBAC: Adding query filter params (role.id=0044294fcceb4b6ba0cb27567f2761d4, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user.id=f9bf4be1c5e94c578c2e02b627c30a56) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:21.884579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84bcd2-287f-430a-8635-2e4e5c789941 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:21.886081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84bcd2-287f-430a-8635-2e4e5c789941 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:21.900544 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb14ff2-8000-4751-8c61-ec8d049e9c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:21.902772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb14ff2-8000-4751-8c61-ec8d049e9c7c None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:21.903133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb14ff2-8000-4751-8c61-ec8d049e9c7c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:21.903425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 94/179] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:21 2022] GET /identity/v3/role_assignments?user.id=f9bf4be1c5e94c578c2e02b627c30a56&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=0044294fcceb4b6ba0cb27567f2761d4 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:21.903531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb14ff2-8000-4751-8c61-ec8d049e9c7c None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/users/fa02e6ef015247398a8c719c2d90c7de/roles/74ea4fc9cad8496a831bb325feb84d4c` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:21.905756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb14ff2-8000-4751-8c61-ec8d049e9c7c None admin] RBAC: Authorizing `identity:create_grant(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user_id=fa02e6ef015247398a8c719c2d90c7de, role_id=74ea4fc9cad8496a831bb325feb84d4c)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:21.907380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb14ff2-8000-4751-8c61-ec8d049e9c7c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:21.924143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 86/180] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:34:21 2022] PUT /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/users/fa02e6ef015247398a8c719c2d90c7de/roles/74ea4fc9cad8496a831bb325feb84d4c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:22.315831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07af02-fab0-42ab-86eb-980e872f45c7 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:22.315831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07af02-fab0-42ab-86eb-980e872f45c7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:22.316119 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07af02-fab0-42ab-86eb-980e872f45c7 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:22.317680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 95/181] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:22.324669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d9b6-15a1-41ec-9425-90c05dada51a None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:22.324969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d9b6-15a1-41ec-9425-90c05dada51a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:22.325258 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d9b6-15a1-41ec-9425-90c05dada51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:22.344590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-4945d9b6-15a1-41ec-9425-90c05dada51a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:22.346258 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4945d9b6-15a1-41ec-9425-90c05dada51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:22.458314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 87/182] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:22.484943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07af02-fab0-42ab-86eb-980e872f45c7 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:22.584813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afaa89-ece1-413d-a754-b29efbbebdc0 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:22.585607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afaa89-ece1-413d-a754-b29efbbebdc0 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:22.586096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afaa89-ece1-413d-a754-b29efbbebdc0 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:22.587814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 88/183] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:34:22.594846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f940b-c4c2-47e1-b800-923401867ee0 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:22.596394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f940b-c4c2-47e1-b800-923401867ee0 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:22.597140 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f940b-c4c2-47e1-b800-923401867ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:22.615776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07af02-fab0-42ab-86eb-980e872f45c7 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:22.622254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-047f940b-c4c2-47e1-b800-923401867ee0 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:22.625139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-047f940b-c4c2-47e1-b800-923401867ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:22.672020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c7cd7-d97a-4c00-96af-4a722e746895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:22.673647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c7cd7-d97a-4c00-96af-4a722e746895 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:22.674195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c7cd7-d97a-4c00-96af-4a722e746895 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:22.674630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c7cd7-d97a-4c00-96af-4a722e746895 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:22.690115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c7cd7-d97a-4c00-96af-4a722e746895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-bf4c7cd7-d97a-4c00-96af-4a722e746895 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:22.705374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:22.706781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:22.707707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:22.715755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 96/184] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:34:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:22.718538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c7cd7-d97a-4c00-96af-4a722e746895 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:22.728374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 89/185] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:22.745682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:22.754267 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f940b-c4c2-47e1-b800-923401867ee0 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:22.793333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:22.843758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c7cd7-d97a-4c00-96af-4a722e746895 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:22.882939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64488c8b-8624-451f-a8e6-a7441ab497de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:22.883878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64488c8b-8624-451f-a8e6-a7441ab497de None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:22.884377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64488c8b-8624-451f-a8e6-a7441ab497de None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:22.884828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64488c8b-8624-451f-a8e6-a7441ab497de None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:22.886236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64488c8b-8624-451f-a8e6-a7441ab497de None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:22.886749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64488c8b-8624-451f-a8e6-a7441ab497de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:22.888338 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64488c8b-8624-451f-a8e6-a7441ab497de None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:22.897995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f940b-c4c2-47e1-b800-923401867ee0 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:22.898965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 97/186] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:22.904872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64488c8b-8624-451f-a8e6-a7441ab497de None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:22.940472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad12cf-e464-47fd-a41b-562517be177c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:22.941340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad12cf-e464-47fd-a41b-562517be177c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:22.941657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad12cf-e464-47fd-a41b-562517be177c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:22.942270 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad12cf-e464-47fd-a41b-562517be177c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:22.955294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad12cf-e464-47fd-a41b-562517be177c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-d8ad12cf-e464-47fd-a41b-562517be177c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:22.963675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:22.964715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:22.965822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 90/187] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:22.966953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff7b41-6033-4767-9941-b1aa310805d7 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:22.966953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff7b41-6033-4767-9941-b1aa310805d7 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:22.967573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff7b41-6033-4767-9941-b1aa310805d7 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:22.968678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 91/188] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:22.974075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff7b41-6033-4767-9941-b1aa310805d7 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:23.011554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:23.047417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64488c8b-8624-451f-a8e6-a7441ab497de None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:23.083146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:23.087104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9dbb0-1e70-4ed0-89cf-f29bfede67b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:23.088364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9dbb0-1e70-4ed0-89cf-f29bfede67b6 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:23.088862 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9dbb0-1e70-4ed0-89cf-f29bfede67b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:23.089270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9dbb0-1e70-4ed0-89cf-f29bfede67b6 None admin] PATH_INFO: `/v3/users/dc3052916c72484b8edf2b28a89a745e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:23.099203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff7b41-6033-4767-9941-b1aa310805d7 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:23.107031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9dbb0-1e70-4ed0-89cf-f29bfede67b6 None admin] RBAC: Authorizing `identity:get_user(user_id=dc3052916c72484b8edf2b28a89a745e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:23.109046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9dbb0-1e70-4ed0-89cf-f29bfede67b6 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:23.111161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 98/189] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:22 2022] GET /identity/v3/users/dc3052916c72484b8edf2b28a89a745e => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:23.112347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb12642-aeb2-43d2-9f0a-2f9af3483948 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:23.112640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb12642-aeb2-43d2-9f0a-2f9af3483948 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:23.112920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb12642-aeb2-43d2-9f0a-2f9af3483948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:23.134520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-3bb12642-aeb2-43d2-9f0a-2f9af3483948 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:23.136422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3bb12642-aeb2-43d2-9f0a-2f9af3483948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:23.146291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f13ad-20fa-4e03-82f1-0c897bfce3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:23.147126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f13ad-20fa-4e03-82f1-0c897bfce3be None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:23.147229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f13ad-20fa-4e03-82f1-0c897bfce3be None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:23.147498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f13ad-20fa-4e03-82f1-0c897bfce3be None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:23.148262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f13ad-20fa-4e03-82f1-0c897bfce3be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:23.148572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f13ad-20fa-4e03-82f1-0c897bfce3be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:23.150003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f13ad-20fa-4e03-82f1-0c897bfce3be None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:23.158788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 92/190] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:23.163869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f13ad-20fa-4e03-82f1-0c897bfce3be None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:23.237388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 99/191] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:23.237388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb12642-aeb2-43d2-9f0a-2f9af3483948 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:23.290016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f13ad-20fa-4e03-82f1-0c897bfce3be None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:23.309907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:23.334184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662976a-19bf-423f-89db-76959d26d703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:23.334874 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662976a-19bf-423f-89db-76959d26d703 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:23.335165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662976a-19bf-423f-89db-76959d26d703 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:23.335447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662976a-19bf-423f-89db-76959d26d703 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:23.355956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb12642-aeb2-43d2-9f0a-2f9af3483948 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:23.359257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:23.412287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6457f-bfb4-4c91-9a24-da32d809f2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:23.413235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6457f-bfb4-4c91-9a24-da32d809f2d1 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:23.413617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6457f-bfb4-4c91-9a24-da32d809f2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:23.414052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6457f-bfb4-4c91-9a24-da32d809f2d1 None admin] PATH_INFO: `/v3/users/dc3052916c72484b8edf2b28a89a745e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:23.415325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6457f-bfb4-4c91-9a24-da32d809f2d1 None admin] RBAC: Authorizing `identity:get_user(user_id=dc3052916c72484b8edf2b28a89a745e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:23.416786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6457f-bfb4-4c91-9a24-da32d809f2d1 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:23.423753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 100/192] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:23 2022] GET /identity/v3/users/dc3052916c72484b8edf2b28a89a745e => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:23.425525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6457f-bfb4-4c91-9a24-da32d809f2d1 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:23.440582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662976a-19bf-423f-89db-76959d26d703 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:23.442089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662976a-19bf-423f-89db-76959d26d703 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:23.545631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6457f-bfb4-4c91-9a24-da32d809f2d1 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:23.572099 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 93/193] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:23.572956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662976a-19bf-423f-89db-76959d26d703 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:23.595761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d37af-db9c-464e-8010-f27c543a21cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:23.596558 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d37af-db9c-464e-8010-f27c543a21cc None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:23.596852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d37af-db9c-464e-8010-f27c543a21cc None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:23.597138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d37af-db9c-464e-8010-f27c543a21cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:23.598382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d37af-db9c-464e-8010-f27c543a21cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:23.599827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d37af-db9c-464e-8010-f27c543a21cc None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:23.601518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 101/194] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:23.602365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d37af-db9c-464e-8010-f27c543a21cc None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:23.605614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:23.633975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:23.686414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662976a-19bf-423f-89db-76959d26d703 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:23.714384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d37af-db9c-464e-8010-f27c543a21cc None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:23.725519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3791ab1-4a3d-4b36-b85b-06ec625ba385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:23.726186 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3791ab1-4a3d-4b36-b85b-06ec625ba385 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:23.726480 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3791ab1-4a3d-4b36-b85b-06ec625ba385 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:23.727075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3791ab1-4a3d-4b36-b85b-06ec625ba385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:23.750579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5935f11-a08a-4aa8-9d9c-74b2a91730fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:23.751547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935f11-a08a-4aa8-9d9c-74b2a91730fd None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:23.751833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935f11-a08a-4aa8-9d9c-74b2a91730fd None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:23.752217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935f11-a08a-4aa8-9d9c-74b2a91730fd None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:23.753504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5935f11-a08a-4aa8-9d9c-74b2a91730fd None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:23.755295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5935f11-a08a-4aa8-9d9c-74b2a91730fd None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:23.762475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 102/195] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:23 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:23.762475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5935f11-a08a-4aa8-9d9c-74b2a91730fd None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:23.803921 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3791ab1-4a3d-4b36-b85b-06ec625ba385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:23.805429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3791ab1-4a3d-4b36-b85b-06ec625ba385 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:23.837566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:23.867041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:23.876759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5935f11-a08a-4aa8-9d9c-74b2a91730fd None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:23.914679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ea2c6-ed73-4096-9f31-73ec87695bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:23.916015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ea2c6-ed73-4096-9f31-73ec87695bfb None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:23.916745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ea2c6-ed73-4096-9f31-73ec87695bfb None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:23.917105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ea2c6-ed73-4096-9f31-73ec87695bfb None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:23.917792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ea2c6-ed73-4096-9f31-73ec87695bfb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:23.920707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ea2c6-ed73-4096-9f31-73ec87695bfb None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:23.925808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 94/196] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:23.927563 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3791ab1-4a3d-4b36-b85b-06ec625ba385 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:23.946298 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 103/197] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:34:23 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:34:23.947139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ea2c6-ed73-4096-9f31-73ec87695bfb None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:24.064833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3791ab1-4a3d-4b36-b85b-06ec625ba385 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:24.072901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ea2c6-ed73-4096-9f31-73ec87695bfb None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:24.104318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:24.117863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54e0bd-fee3-4e9c-824a-df8863f4da60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:24.118140 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d59868-cdaa-415f-8ab7-333eee5737ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:24.118968 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d59868-cdaa-415f-8ab7-333eee5737ed None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:24.119327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d59868-cdaa-415f-8ab7-333eee5737ed None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:24.119650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54e0bd-fee3-4e9c-824a-df8863f4da60 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:24.119753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d59868-cdaa-415f-8ab7-333eee5737ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:24.120211 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54e0bd-fee3-4e9c-824a-df8863f4da60 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:24.120676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d59868-cdaa-415f-8ab7-333eee5737ed None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=0d8860ce338747a5ba3d5903b48cc986, user.id=dc3052916c72484b8edf2b28a89a745e) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:24.120969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54e0bd-fee3-4e9c-824a-df8863f4da60 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:24.121069 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d59868-cdaa-415f-8ab7-333eee5737ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:24.122564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54e0bd-fee3-4e9c-824a-df8863f4da60 None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:24.123450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d59868-cdaa-415f-8ab7-333eee5737ed None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:24.125077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54e0bd-fee3-4e9c-824a-df8863f4da60 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:24.127433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 104/198] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:23 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:24.132394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 95/199] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:23 2022] GET /identity/v3/role_assignments?user.id=dc3052916c72484b8edf2b28a89a745e&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=85420c68300642aabfd730fb8fef8347 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:24.136931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54e0bd-fee3-4e9c-824a-df8863f4da60 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:24.261594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54e0bd-fee3-4e9c-824a-df8863f4da60 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:24.294515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd63dd4-bb0c-4a91-b70d-1edc6af26a2c None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:24.294860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd63dd4-bb0c-4a91-b70d-1edc6af26a2c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:24.295127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd63dd4-bb0c-4a91-b70d-1edc6af26a2c None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:24.296551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 96/200] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:24.302927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c3c37-0264-4868-b1cf-621b21cd9dd1 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:24.303210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c3c37-0264-4868-b1cf-621b21cd9dd1 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:24.303490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c3c37-0264-4868-b1cf-621b21cd9dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:24.307571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870ea27-4080-4f27-8aa7-dc0478b4bdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:24.308245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870ea27-4080-4f27-8aa7-dc0478b4bdc5 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:24.308531 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870ea27-4080-4f27-8aa7-dc0478b4bdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:24.308825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870ea27-4080-4f27-8aa7-dc0478b4bdc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:24.309707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870ea27-4080-4f27-8aa7-dc0478b4bdc5 None admin] RBAC: Adding query filter params (role.id=0044294fcceb4b6ba0cb27567f2761d4, scope.project.id=0d8860ce338747a5ba3d5903b48cc986, user.id=dc3052916c72484b8edf2b28a89a745e) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:24.310779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870ea27-4080-4f27-8aa7-dc0478b4bdc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:24.312166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870ea27-4080-4f27-8aa7-dc0478b4bdc5 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:24.319665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 105/201] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:24 2022] GET /identity/v3/role_assignments?user.id=dc3052916c72484b8edf2b28a89a745e&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=0044294fcceb4b6ba0cb27567f2761d4 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:24.329972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a21c3c37-0264-4868-b1cf-621b21cd9dd1 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:24.332757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a21c3c37-0264-4868-b1cf-621b21cd9dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:24.345199 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:24.431212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 97/202] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:24.457758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870ea27-4080-4f27-8aa7-dc0478b4bdc5 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:24.535505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:24.588705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870ea27-4080-4f27-8aa7-dc0478b4bdc5 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:24.634425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76852e41-f799-484e-b0ce-169187726f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:24.635779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76852e41-f799-484e-b0ce-169187726f8f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:24.636268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76852e41-f799-484e-b0ce-169187726f8f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:24.636757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76852e41-f799-484e-b0ce-169187726f8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:24.648035 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76852e41-f799-484e-b0ce-169187726f8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-76852e41-f799-484e-b0ce-169187726f8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:24.660549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:24.661513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:24.662280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:24.665001 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 106/203] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:24.674506 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c3c37-0264-4868-b1cf-621b21cd9dd1 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:24.814327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c3c37-0264-4868-b1cf-621b21cd9dd1 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:24.860882 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd9b98-d532-4e3f-9545-3dd317ed496d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:24.861607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd9b98-d532-4e3f-9545-3dd317ed496d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:24.861892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd9b98-d532-4e3f-9545-3dd317ed496d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:24.862172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd9b98-d532-4e3f-9545-3dd317ed496d None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:24.863002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd9b98-d532-4e3f-9545-3dd317ed496d None admin] RBAC: Adding query filter params (name=reader) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:24.863315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd9b98-d532-4e3f-9545-3dd317ed496d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:24.864716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd9b98-d532-4e3f-9545-3dd317ed496d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:24.874851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 98/204] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:24.882444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76852e41-f799-484e-b0ce-169187726f8f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:25.056760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76852e41-f799-484e-b0ce-169187726f8f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:25.084110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:25.110138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e0e76-9d0f-4926-ba5f-4a432712196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:25.111028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0e76-9d0f-4926-ba5f-4a432712196a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:25.111315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0e76-9d0f-4926-ba5f-4a432712196a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:25.111594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e0e76-9d0f-4926-ba5f-4a432712196a None admin] PATH_INFO: `/v3/users/fa02e6ef015247398a8c719c2d90c7de` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:25.112801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e0e76-9d0f-4926-ba5f-4a432712196a None admin] RBAC: Authorizing `identity:get_user(user_id=fa02e6ef015247398a8c719c2d90c7de)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:25.114358 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e0e76-9d0f-4926-ba5f-4a432712196a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:25.115999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 107/205] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:24 2022] GET /identity/v3/users/fa02e6ef015247398a8c719c2d90c7de => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:25.121821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd9b98-d532-4e3f-9545-3dd317ed496d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:25.140101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:25.285377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd9b98-d532-4e3f-9545-3dd317ed496d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:25.333110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19689dfc-5c99-48f0-96c3-81b82048e475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:25.333795 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689dfc-5c99-48f0-96c3-81b82048e475 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:25.334083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689dfc-5c99-48f0-96c3-81b82048e475 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:25.334416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689dfc-5c99-48f0-96c3-81b82048e475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:25.432981 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19689dfc-5c99-48f0-96c3-81b82048e475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:25.434478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19689dfc-5c99-48f0-96c3-81b82048e475 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:25.447753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:25.575491 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 99/206] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:25.581376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e0e76-9d0f-4926-ba5f-4a432712196a None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:25.705638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e0e76-9d0f-4926-ba5f-4a432712196a None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:25.756893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e3f48-42e8-4d5a-be1d-192de1cc84d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:25.758154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e3f48-42e8-4d5a-be1d-192de1cc84d2 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:25.758681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e3f48-42e8-4d5a-be1d-192de1cc84d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:25.759155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e3f48-42e8-4d5a-be1d-192de1cc84d2 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:25.760551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e3f48-42e8-4d5a-be1d-192de1cc84d2 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:25.762229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e3f48-42e8-4d5a-be1d-192de1cc84d2 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:25.764653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 108/207] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:25 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:25.770470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19689dfc-5c99-48f0-96c3-81b82048e475 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:25.904813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19689dfc-5c99-48f0-96c3-81b82048e475 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:25.912387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:25.940528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa470ce2-4b84-4ee7-968e-7512a0ac1df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:25.941192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470ce2-4b84-4ee7-968e-7512a0ac1df0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:25.941506 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470ce2-4b84-4ee7-968e-7512a0ac1df0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:25.941786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470ce2-4b84-4ee7-968e-7512a0ac1df0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:25.942627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa470ce2-4b84-4ee7-968e-7512a0ac1df0 None admin] RBAC: Adding query filter params (role.id=74ea4fc9cad8496a831bb325feb84d4c, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc, user.id=fa02e6ef015247398a8c719c2d90c7de) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:25.942940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa470ce2-4b84-4ee7-968e-7512a0ac1df0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:25.944384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa470ce2-4b84-4ee7-968e-7512a0ac1df0 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:25.956713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 100/208] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:25 2022] GET /identity/v3/role_assignments?user.id=fa02e6ef015247398a8c719c2d90c7de&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=74ea4fc9cad8496a831bb325feb84d4c => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:26.687645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6483c-2569-42c3-85ac-9ee462f6f283 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:26.687645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6483c-2569-42c3-85ac-9ee462f6f283 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:26.687645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6483c-2569-42c3-85ac-9ee462f6f283 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:26.687645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 109/209] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:26.694079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc852b8-5c21-4f99-b880-0adc644f5037 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:26.694626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc852b8-5c21-4f99-b880-0adc644f5037 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:26.695102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc852b8-5c21-4f99-b880-0adc644f5037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:26.720486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-9cc852b8-5c21-4f99-b880-0adc644f5037 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:26.722029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-9cc852b8-5c21-4f99-b880-0adc644f5037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:26.822080 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 101/210] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:26.824165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74de0f-886e-4887-b6cb-b33b90306ffc None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:26.824456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74de0f-886e-4887-b6cb-b33b90306ffc None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:26.824738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74de0f-886e-4887-b6cb-b33b90306ffc None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:26.826857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 110/211] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:26.835007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244c518-29bc-4c57-a373-3a59682745bc None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:26.835007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244c518-29bc-4c57-a373-3a59682745bc None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:26.835007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244c518-29bc-4c57-a373-3a59682745bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:26.846154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74de0f-886e-4887-b6cb-b33b90306ffc None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:26.853751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-8244c518-29bc-4c57-a373-3a59682745bc None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:26.855157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8244c518-29bc-4c57-a373-3a59682745bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:26.916500 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:26.942391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 102/212] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:26.970233 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244c518-29bc-4c57-a373-3a59682745bc None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:26.970557 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74de0f-886e-4887-b6cb-b33b90306ffc None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:27.004782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:27.005341 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69f327-b02b-477f-a614-5d9632367a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:27.006472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f327-b02b-477f-a614-5d9632367a21 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.006472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f327-b02b-477f-a614-5d9632367a21 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.006699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f327-b02b-477f-a614-5d9632367a21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.017972 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69f327-b02b-477f-a614-5d9632367a21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-fa69f327-b02b-477f-a614-5d9632367a21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:27.025745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:27.026646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:27.027426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 111/213] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:34:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:34:27.028742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c7f8f-7bed-43bc-bd79-fe7b5a54956f None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.028742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c7f8f-7bed-43bc-bd79-fe7b5a54956f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.028742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c7f8f-7bed-43bc-bd79-fe7b5a54956f None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.030197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 112/214] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:27.034684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c7f8f-7bed-43bc-bd79-fe7b5a54956f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:27.085569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244c518-29bc-4c57-a373-3a59682745bc None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:27.120110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0781ca-5d51-4ea7-bfa3-bd5c929ef9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:27.120908 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0781ca-5d51-4ea7-bfa3-bd5c929ef9a3 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.121194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0781ca-5d51-4ea7-bfa3-bd5c929ef9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.121499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0781ca-5d51-4ea7-bfa3-bd5c929ef9a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.122677 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0781ca-5d51-4ea7-bfa3-bd5c929ef9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:27.124705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0781ca-5d51-4ea7-bfa3-bd5c929ef9a3 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:27.126381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 103/215] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:27.127523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91877941-5c55-4fb2-8220-1eaafd177e14 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.127814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91877941-5c55-4fb2-8220-1eaafd177e14 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.128096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91877941-5c55-4fb2-8220-1eaafd177e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.137534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c7f8f-7bed-43bc-bd79-fe7b5a54956f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:27.143792 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-91877941-5c55-4fb2-8220-1eaafd177e14 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:27.145003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-91877941-5c55-4fb2-8220-1eaafd177e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:27.156039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:27.178950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360291f6-862f-4c75-8263-59d44d812ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:27.179615 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360291f6-862f-4c75-8263-59d44d812ec0 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.179891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360291f6-862f-4c75-8263-59d44d812ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.180162 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360291f6-862f-4c75-8263-59d44d812ec0 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.180908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360291f6-862f-4c75-8263-59d44d812ec0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:27.181211 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360291f6-862f-4c75-8263-59d44d812ec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:27.182611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360291f6-862f-4c75-8263-59d44d812ec0 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:27.191887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 113/216] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:27.196691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab31fb-32dc-4120-8854-706245527e97 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.196691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab31fb-32dc-4120-8854-706245527e97 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.196691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab31fb-32dc-4120-8854-706245527e97 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.196691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 114/217] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:27.196691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab31fb-32dc-4120-8854-706245527e97 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:27.229659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 104/218] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:27.230509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91877941-5c55-4fb2-8220-1eaafd177e14 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:27.262648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:27.299486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab31fb-32dc-4120-8854-706245527e97 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:27.333126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7b507-a0b0-4249-bf51-c08e013bb608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:27.333831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7b507-a0b0-4249-bf51-c08e013bb608 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.334116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7b507-a0b0-4249-bf51-c08e013bb608 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.334394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7b507-a0b0-4249-bf51-c08e013bb608 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.335104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7b507-a0b0-4249-bf51-c08e013bb608 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:27.336535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7b507-a0b0-4249-bf51-c08e013bb608 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:27.339637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91877941-5c55-4fb2-8220-1eaafd177e14 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:27.359693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 115/219] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:34:27 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:34:27.360967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1423fb6-291e-4718-b684-7089ef3f3661 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.361257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1423fb6-291e-4718-b684-7089ef3f3661 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.361577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1423fb6-291e-4718-b684-7089ef3f3661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.379651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-c1423fb6-291e-4718-b684-7089ef3f3661 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:27.380966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c1423fb6-291e-4718-b684-7089ef3f3661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:27.393979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1adff1-281e-4cad-9a63-1edb71f285ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:27.394887 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1adff1-281e-4cad-9a63-1edb71f285ca None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.395169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1adff1-281e-4cad-9a63-1edb71f285ca None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.395439 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1adff1-281e-4cad-9a63-1edb71f285ca None admin] PATH_INFO: `/v3/users/dc3052916c72484b8edf2b28a89a745e` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.396749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1adff1-281e-4cad-9a63-1edb71f285ca None admin] RBAC: Authorizing `identity:get_user(user_id=dc3052916c72484b8edf2b28a89a745e)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:27.398280 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1adff1-281e-4cad-9a63-1edb71f285ca None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:27.405073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 105/220] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:27 2022] GET /identity/v3/users/dc3052916c72484b8edf2b28a89a745e => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:27.405073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1adff1-281e-4cad-9a63-1edb71f285ca None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:27.416926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:27.470062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 116/221] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:27.470677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1423fb6-291e-4718-b684-7089ef3f3661 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:27.522521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:27.530767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1adff1-281e-4cad-9a63-1edb71f285ca None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:27.570642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5589da-2b7b-4496-8b07-e1c4df2e7894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:27.571444 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5589da-2b7b-4496-8b07-e1c4df2e7894 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.571739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5589da-2b7b-4496-8b07-e1c4df2e7894 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.571988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5589da-2b7b-4496-8b07-e1c4df2e7894 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.586284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5589da-2b7b-4496-8b07-e1c4df2e7894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-3a5589da-2b7b-4496-8b07-e1c4df2e7894 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:27.595741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:27.596683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:27.597556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 106/222] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:27.598194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5589da-2b7b-4496-8b07-e1c4df2e7894 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:27.605825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1423fb6-291e-4718-b684-7089ef3f3661 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:27.655554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c699d89-19e5-447b-b0d7-bb5d9de87e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:27.656899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c699d89-19e5-447b-b0d7-bb5d9de87e8b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.656899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c699d89-19e5-447b-b0d7-bb5d9de87e8b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.657071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c699d89-19e5-447b-b0d7-bb5d9de87e8b None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.658201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c699d89-19e5-447b-b0d7-bb5d9de87e8b None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:27.659689 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c699d89-19e5-447b-b0d7-bb5d9de87e8b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:27.661471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 117/223] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:27 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:27.662301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c699d89-19e5-447b-b0d7-bb5d9de87e8b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:27.699912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:27.732697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5589da-2b7b-4496-8b07-e1c4df2e7894 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:27.770316 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28086e8c-f563-460c-a8fa-813c4bd27ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:27.771176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28086e8c-f563-460c-a8fa-813c4bd27ba0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.771467 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28086e8c-f563-460c-a8fa-813c4bd27ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.771845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28086e8c-f563-460c-a8fa-813c4bd27ba0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.783244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28086e8c-f563-460c-a8fa-813c4bd27ba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-28086e8c-f563-460c-a8fa-813c4bd27ba0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:27.792033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:27.792970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 107/224] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:27.794170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28086e8c-f563-460c-a8fa-813c4bd27ba0 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:27.799598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c699d89-19e5-447b-b0d7-bb5d9de87e8b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:27.829792 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:27.844842 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0bf9a-d40e-481b-bbf4-c16729ed72c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:27.845696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0bf9a-d40e-481b-bbf4-c16729ed72c3 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.846086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0bf9a-d40e-481b-bbf4-c16729ed72c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.846573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0bf9a-d40e-481b-bbf4-c16729ed72c3 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.847666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0bf9a-d40e-481b-bbf4-c16729ed72c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:27.848041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0bf9a-d40e-481b-bbf4-c16729ed72c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:27.850052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0bf9a-d40e-481b-bbf4-c16729ed72c3 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:27.862754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 118/225] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:27.868582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0bf9a-d40e-481b-bbf4-c16729ed72c3 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:27.921229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28086e8c-f563-460c-a8fa-813c4bd27ba0 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:27.958111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c2d34-973b-4d7d-8a85-b25cc1ad88bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:27.959105 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c2d34-973b-4d7d-8a85-b25cc1ad88bf None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:27.959456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c2d34-973b-4d7d-8a85-b25cc1ad88bf None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:27.959801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c2d34-973b-4d7d-8a85-b25cc1ad88bf None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986/users/dc3052916c72484b8edf2b28a89a745e/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:27.962390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c2d34-973b-4d7d-8a85-b25cc1ad88bf None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8860ce338747a5ba3d5903b48cc986, user_id=dc3052916c72484b8edf2b28a89a745e, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:27.963920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c2d34-973b-4d7d-8a85-b25cc1ad88bf None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:27.981440 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 108/226] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:34:27 2022] PUT /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986/users/dc3052916c72484b8edf2b28a89a745e/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:27.982915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c2d34-973b-4d7d-8a85-b25cc1ad88bf None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:27.993405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:28.012158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0bf9a-d40e-481b-bbf4-c16729ed72c3 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:28.082353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94208d-4b16-461a-803f-604d10ad578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:28.083383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94208d-4b16-461a-803f-604d10ad578d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:28.083846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94208d-4b16-461a-803f-604d10ad578d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:28.085917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94208d-4b16-461a-803f-604d10ad578d None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:28.086945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94208d-4b16-461a-803f-604d10ad578d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:28.087469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94208d-4b16-461a-803f-604d10ad578d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:28.089161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94208d-4b16-461a-803f-604d10ad578d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:28.103861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 119/227] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:28.111458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94208d-4b16-461a-803f-604d10ad578d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:28.147440 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:28.190123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c2d34-973b-4d7d-8a85-b25cc1ad88bf None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:28.229243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2074bb-c998-44ec-bc33-041369ea1f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:28.230464 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2074bb-c998-44ec-bc33-041369ea1f8b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:28.230926 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2074bb-c998-44ec-bc33-041369ea1f8b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:28.231376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2074bb-c998-44ec-bc33-041369ea1f8b None admin] PATH_INFO: `/v3/users/ae8e8983b85348afb0e1c8712cbe7255` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:28.246199 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2074bb-c998-44ec-bc33-041369ea1f8b None admin] RBAC: Authorizing `identity:get_user(user_id=ae8e8983b85348afb0e1c8712cbe7255)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:28.248495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2074bb-c998-44ec-bc33-041369ea1f8b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:28.251972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 109/228] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:27 2022] GET /identity/v3/users/ae8e8983b85348afb0e1c8712cbe7255 => generated 297 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:28.258976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2074bb-c998-44ec-bc33-041369ea1f8b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:28.260652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94208d-4b16-461a-803f-604d10ad578d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:28.298093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104d62b-b650-4901-8fd7-050aa5b81eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:28.298894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104d62b-b650-4901-8fd7-050aa5b81eda None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:28.299247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104d62b-b650-4901-8fd7-050aa5b81eda None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:28.299590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104d62b-b650-4901-8fd7-050aa5b81eda None admin] PATH_INFO: `/v3/users/dc3052916c72484b8edf2b28a89a745e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:28.300944 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104d62b-b650-4901-8fd7-050aa5b81eda None admin] RBAC: Authorizing `identity:get_user(user_id=dc3052916c72484b8edf2b28a89a745e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:28.302555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104d62b-b650-4901-8fd7-050aa5b81eda None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:28.304703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 120/229] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:28 2022] GET /identity/v3/users/dc3052916c72484b8edf2b28a89a745e => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:28.309818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104d62b-b650-4901-8fd7-050aa5b81eda None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:28.370625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2074bb-c998-44ec-bc33-041369ea1f8b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:28.386917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:28.409084 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b031c-fcb9-4c03-9ada-00f707304b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:28.410333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b031c-fcb9-4c03-9ada-00f707304b23 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:28.410766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b031c-fcb9-4c03-9ada-00f707304b23 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:28.411187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b031c-fcb9-4c03-9ada-00f707304b23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:28.423095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:28.483207 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b031c-fcb9-4c03-9ada-00f707304b23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:28.484905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b031c-fcb9-4c03-9ada-00f707304b23 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:28.602321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104d62b-b650-4901-8fd7-050aa5b81eda None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:28.649449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 110/230] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:34:28.657059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b031c-fcb9-4c03-9ada-00f707304b23 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:28.671060 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83523a62-cd49-4209-bb1b-9388718862f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:28.672037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83523a62-cd49-4209-bb1b-9388718862f7 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:28.672393 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83523a62-cd49-4209-bb1b-9388718862f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:28.672933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83523a62-cd49-4209-bb1b-9388718862f7 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:28.678111 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83523a62-cd49-4209-bb1b-9388718862f7 None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:28.679641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83523a62-cd49-4209-bb1b-9388718862f7 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:28.681809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 121/231] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:28 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:28.687585 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83523a62-cd49-4209-bb1b-9388718862f7 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:28.694121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:28.768143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b031c-fcb9-4c03-9ada-00f707304b23 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:28.806543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861c22e-a82e-4a71-b81f-1a8d36ecf040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:28.807682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861c22e-a82e-4a71-b81f-1a8d36ecf040 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:28.808427 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861c22e-a82e-4a71-b81f-1a8d36ecf040 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:28.808851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861c22e-a82e-4a71-b81f-1a8d36ecf040 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:28.810891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861c22e-a82e-4a71-b81f-1a8d36ecf040 None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:28.812802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861c22e-a82e-4a71-b81f-1a8d36ecf040 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:28.815418 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 111/232] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:28 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:28.820905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861c22e-a82e-4a71-b81f-1a8d36ecf040 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:28.889028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:28.900219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83523a62-cd49-4209-bb1b-9388718862f7 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:28.937141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011977f-385c-4e8b-b998-fe243b7e879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:28.938391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011977f-385c-4e8b-b998-fe243b7e879c None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:28.938851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011977f-385c-4e8b-b998-fe243b7e879c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:28.939739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011977f-385c-4e8b-b998-fe243b7e879c None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986/users/dc3052916c72484b8edf2b28a89a745e/roles/0044294fcceb4b6ba0cb27567f2761d4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:28.948435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011977f-385c-4e8b-b998-fe243b7e879c None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8860ce338747a5ba3d5903b48cc986, user_id=dc3052916c72484b8edf2b28a89a745e, role_id=0044294fcceb4b6ba0cb27567f2761d4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:28.950404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011977f-385c-4e8b-b998-fe243b7e879c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:28.969336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 122/233] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:34:28 2022] PUT /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986/users/dc3052916c72484b8edf2b28a89a745e/roles/0044294fcceb4b6ba0cb27567f2761d4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:34:29.008414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861c22e-a82e-4a71-b81f-1a8d36ecf040 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:29.020383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:29.046716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812d7c7-c1fb-48d9-94d1-5df3c6bc3d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:29.047680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812d7c7-c1fb-48d9-94d1-5df3c6bc3d7a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:29.048243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812d7c7-c1fb-48d9-94d1-5df3c6bc3d7a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:29.048693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812d7c7-c1fb-48d9-94d1-5df3c6bc3d7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:29.049740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812d7c7-c1fb-48d9-94d1-5df3c6bc3d7a None admin] RBAC: Adding query filter params (role.id=d94de411ec0a40be92c04950e6edcac7, scope.project.id=0d8860ce338747a5ba3d5903b48cc986, user.id=ae8e8983b85348afb0e1c8712cbe7255) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:29.050231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812d7c7-c1fb-48d9-94d1-5df3c6bc3d7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:29.051853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812d7c7-c1fb-48d9-94d1-5df3c6bc3d7a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:29.058640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 112/234] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:28 2022] GET /identity/v3/role_assignments?user.id=ae8e8983b85348afb0e1c8712cbe7255&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:29.844906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b80c4-7f3f-4a57-b3cb-52e2150fd2b4 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:29.845191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b80c4-7f3f-4a57-b3cb-52e2150fd2b4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:29.845703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b80c4-7f3f-4a57-b3cb-52e2150fd2b4 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:29.847451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 123/235] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:29.853246 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa09ad-0e07-4f63-b1cf-d5f8ac3f1ef4 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:29.853676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa09ad-0e07-4f63-b1cf-d5f8ac3f1ef4 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:29.854032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa09ad-0e07-4f63-b1cf-d5f8ac3f1ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:29.878388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-e5aa09ad-0e07-4f63-b1cf-d5f8ac3f1ef4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:29.879606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e5aa09ad-0e07-4f63-b1cf-d5f8ac3f1ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:29.972782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 113/236] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:30.006270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b80c4-7f3f-4a57-b3cb-52e2150fd2b4 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:30.008210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fea7f8-bef2-46e9-80b7-1c4a492d1689 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:30.008497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fea7f8-bef2-46e9-80b7-1c4a492d1689 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:30.009392 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fea7f8-bef2-46e9-80b7-1c4a492d1689 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:30.010568 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 114/237] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:30.023747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5888cc0-4d91-4922-9e2c-8429ba9abc88 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:30.024113 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5888cc0-4d91-4922-9e2c-8429ba9abc88 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:30.024360 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5888cc0-4d91-4922-9e2c-8429ba9abc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:30.042387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a5888cc0-4d91-4922-9e2c-8429ba9abc88 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:30.043604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a5888cc0-4d91-4922-9e2c-8429ba9abc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:30.143977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 115/238] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:30.198797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5888cc0-4d91-4922-9e2c-8429ba9abc88 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:30.199053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b80c4-7f3f-4a57-b3cb-52e2150fd2b4 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:30.236893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3993e17-5af2-45e5-9894-5404a13719d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:30.237848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3993e17-5af2-45e5-9894-5404a13719d4 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:30.238133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3993e17-5af2-45e5-9894-5404a13719d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:30.238408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3993e17-5af2-45e5-9894-5404a13719d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:30.250266 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3993e17-5af2-45e5-9894-5404a13719d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-f3993e17-5af2-45e5-9894-5404a13719d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:30.258378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:30.259743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:30.261019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:30.262034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:30.262140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 124/239] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:34:30.269406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3993e17-5af2-45e5-9894-5404a13719d4 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:30.283735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:30.381602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5888cc0-4d91-4922-9e2c-8429ba9abc88 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:30.421100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd85979-b1e4-4c51-9976-9abd005585dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:30.421862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd85979-b1e4-4c51-9976-9abd005585dc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:30.422142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd85979-b1e4-4c51-9976-9abd005585dc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:30.422527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd85979-b1e4-4c51-9976-9abd005585dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:30.423863 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd85979-b1e4-4c51-9976-9abd005585dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:30.425531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd85979-b1e4-4c51-9976-9abd005585dc None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:30.427595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 116/240] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:30.433101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd85979-b1e4-4c51-9976-9abd005585dc None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:30.479238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3993e17-5af2-45e5-9894-5404a13719d4 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:30.537763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:30.541917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd85979-b1e4-4c51-9976-9abd005585dc None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:30.567037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba77f91-640e-4906-bf22-486a69a31149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:30.567727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba77f91-640e-4906-bf22-486a69a31149 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:30.568010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba77f91-640e-4906-bf22-486a69a31149 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:30.568284 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba77f91-640e-4906-bf22-486a69a31149 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:30.569048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba77f91-640e-4906-bf22-486a69a31149 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:30.569371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba77f91-640e-4906-bf22-486a69a31149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:30.570752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba77f91-640e-4906-bf22-486a69a31149 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:30.581060 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 125/241] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:30.586893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba77f91-640e-4906-bf22-486a69a31149 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:30.588744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4d174-5c4f-43e9-8226-6a15122ba76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:30.591932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4d174-5c4f-43e9-8226-6a15122ba76d None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:30.592231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4d174-5c4f-43e9-8226-6a15122ba76d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:30.592506 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4d174-5c4f-43e9-8226-6a15122ba76d None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:30.593244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4d174-5c4f-43e9-8226-6a15122ba76d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:30.594728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4d174-5c4f-43e9-8226-6a15122ba76d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:30.637763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 117/242] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:34:30 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:34:30.743189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:30.787461 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba77f91-640e-4906-bf22-486a69a31149 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:30.823778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f9412-8b61-4394-9b7e-14cf16b6a17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:30.824665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f9412-8b61-4394-9b7e-14cf16b6a17d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:30.824947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f9412-8b61-4394-9b7e-14cf16b6a17d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:30.825220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f9412-8b61-4394-9b7e-14cf16b6a17d None admin] PATH_INFO: `/v3/users/1250f110be6f46e990a0cb4ae26a141c` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:30.840474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f9412-8b61-4394-9b7e-14cf16b6a17d None admin] RBAC: Authorizing `identity:get_user(user_id=1250f110be6f46e990a0cb4ae26a141c)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:30.842535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f9412-8b61-4394-9b7e-14cf16b6a17d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:30.844538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 126/243] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:30 2022] GET /identity/v3/users/1250f110be6f46e990a0cb4ae26a141c => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:30.851035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4d174-5c4f-43e9-8226-6a15122ba76d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:30.887949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398e71e-1491-4534-afd3-3358595b8928 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:30.888863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398e71e-1491-4534-afd3-3358595b8928 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:30.889134 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398e71e-1491-4534-afd3-3358595b8928 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:30.890859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 127/244] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:30.896924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af060f0-0334-4049-ad4d-946802be0696 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:30.897268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af060f0-0334-4049-ad4d-946802be0696 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:30.897519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af060f0-0334-4049-ad4d-946802be0696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:30.916578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0af060f0-0334-4049-ad4d-946802be0696 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:30.917836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0af060f0-0334-4049-ad4d-946802be0696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:30.998737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4d174-5c4f-43e9-8226-6a15122ba76d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:31.006745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 128/245] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:31.036246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af060f0-0334-4049-ad4d-946802be0696 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:31.039368 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fbc5b-e172-4258-89e9-5a0c71d65ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:31.040030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fbc5b-e172-4258-89e9-5a0c71d65ddf None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:31.040307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fbc5b-e172-4258-89e9-5a0c71d65ddf None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:31.040640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fbc5b-e172-4258-89e9-5a0c71d65ddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:31.062786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:31.101954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:31.122058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fbc5b-e172-4258-89e9-5a0c71d65ddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:31.126267 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fbc5b-e172-4258-89e9-5a0c71d65ddf None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:31.146323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af060f0-0334-4049-ad4d-946802be0696 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:31.183822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab831e77-42d4-42c0-84a9-c14dc2fc4117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:31.184633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab831e77-42d4-42c0-84a9-c14dc2fc4117 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:31.184906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab831e77-42d4-42c0-84a9-c14dc2fc4117 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:31.185731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab831e77-42d4-42c0-84a9-c14dc2fc4117 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:31.197263 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab831e77-42d4-42c0-84a9-c14dc2fc4117 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-ab831e77-42d4-42c0-84a9-c14dc2fc4117 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:31.207282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:31.208199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:31.208967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:31.215340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 129/246] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:31.215340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab831e77-42d4-42c0-84a9-c14dc2fc4117 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:31.249974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 118/247] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:31.255972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fbc5b-e172-4258-89e9-5a0c71d65ddf None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:31.338121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab831e77-42d4-42c0-84a9-c14dc2fc4117 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:31.357782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:31.382958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:31.383850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fbc5b-e172-4258-89e9-5a0c71d65ddf None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:31.399372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5244b-7e5a-4299-984e-aebd03bd7a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:31.400288 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5244b-7e5a-4299-984e-aebd03bd7a4f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:31.400739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5244b-7e5a-4299-984e-aebd03bd7a4f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:31.401153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5244b-7e5a-4299-984e-aebd03bd7a4f None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:31.403205 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5244b-7e5a-4299-984e-aebd03bd7a4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:31.403721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5244b-7e5a-4299-984e-aebd03bd7a4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:31.405248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5244b-7e5a-4299-984e-aebd03bd7a4f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:31.414876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 130/248] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:31.421038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5244b-7e5a-4299-984e-aebd03bd7a4f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:31.427308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed8796-323b-4af9-9cd6-7fa9159a4eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:31.428379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed8796-323b-4af9-9cd6-7fa9159a4eed None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:31.428761 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed8796-323b-4af9-9cd6-7fa9159a4eed None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:31.429134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed8796-323b-4af9-9cd6-7fa9159a4eed None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:31.430614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed8796-323b-4af9-9cd6-7fa9159a4eed None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:31.433290 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed8796-323b-4af9-9cd6-7fa9159a4eed None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:31.434753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 119/249] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:31 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:31.441266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed8796-323b-4af9-9cd6-7fa9159a4eed None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:31.556130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5244b-7e5a-4299-984e-aebd03bd7a4f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:31.577388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed8796-323b-4af9-9cd6-7fa9159a4eed None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:31.605800 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:31.620853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288f3ea-42d7-4472-8ff7-e47b65ded014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:31.621548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288f3ea-42d7-4472-8ff7-e47b65ded014 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:31.621830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288f3ea-42d7-4472-8ff7-e47b65ded014 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:31.622103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288f3ea-42d7-4472-8ff7-e47b65ded014 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:31.623901 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288f3ea-42d7-4472-8ff7-e47b65ded014 None admin] RBAC: Adding query filter params (role.id=74ea4fc9cad8496a831bb325feb84d4c, scope.project.id=0d8860ce338747a5ba3d5903b48cc986, user.id=1250f110be6f46e990a0cb4ae26a141c) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:31.624225 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288f3ea-42d7-4472-8ff7-e47b65ded014 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:31.625708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288f3ea-42d7-4472-8ff7-e47b65ded014 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:31.640645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 120/250] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:31 2022] GET /identity/v3/role_assignments?user.id=1250f110be6f46e990a0cb4ae26a141c&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=74ea4fc9cad8496a831bb325feb84d4c => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:31.649445 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116684f-216e-4640-ae33-010c9bc2accf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:31.650640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116684f-216e-4640-ae33-010c9bc2accf None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:31.651071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116684f-216e-4640-ae33-010c9bc2accf None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:31.651491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116684f-216e-4640-ae33-010c9bc2accf None admin] PATH_INFO: `/v3/users/dc3052916c72484b8edf2b28a89a745e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:31.654683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116684f-216e-4640-ae33-010c9bc2accf None admin] RBAC: Authorizing `identity:get_user(user_id=dc3052916c72484b8edf2b28a89a745e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:31.656357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116684f-216e-4640-ae33-010c9bc2accf None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:31.658793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 131/251] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:31 2022] GET /identity/v3/users/dc3052916c72484b8edf2b28a89a745e => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:31.664637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288f3ea-42d7-4472-8ff7-e47b65ded014 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:31.680748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907182-620a-4100-b761-495189cf2e22 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:31.681301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907182-620a-4100-b761-495189cf2e22 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:31.681764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907182-620a-4100-b761-495189cf2e22 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:31.683364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 132/252] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:31.689414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ccf8f-3c0e-4054-9143-11ba76998209 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:31.689919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ccf8f-3c0e-4054-9143-11ba76998209 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:31.690335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ccf8f-3c0e-4054-9143-11ba76998209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:31.727706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-743ccf8f-3c0e-4054-9143-11ba76998209 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:31.730437 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-743ccf8f-3c0e-4054-9143-11ba76998209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:31.839042 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288f3ea-42d7-4472-8ff7-e47b65ded014 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:31.851518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:31.890560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 133/253] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:31.905409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3cb6a-382c-43c2-87d2-37601f5d1287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:31.906298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3cb6a-382c-43c2-87d2-37601f5d1287 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:31.906722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3cb6a-382c-43c2-87d2-37601f5d1287 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:31.907138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3cb6a-382c-43c2-87d2-37601f5d1287 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:31.916640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ccf8f-3c0e-4054-9143-11ba76998209 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:31.965435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:31.990588 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3cb6a-382c-43c2-87d2-37601f5d1287 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:31.992050 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3cb6a-382c-43c2-87d2-37601f5d1287 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:32.054885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ccf8f-3c0e-4054-9143-11ba76998209 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:32.102307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 121/254] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:32.103571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994133c-3303-4729-8526-5511e3e671d5 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:32.103855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994133c-3303-4729-8526-5511e3e671d5 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:32.104129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994133c-3303-4729-8526-5511e3e671d5 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:32.105452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 122/255] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:32.109314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994133c-3303-4729-8526-5511e3e671d5 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:32.109995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92e07b-36db-43c0-974c-cd7314cd0c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:32.110803 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92e07b-36db-43c0-974c-cd7314cd0c37 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:32.111083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92e07b-36db-43c0-974c-cd7314cd0c37 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:32.111353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92e07b-36db-43c0-974c-cd7314cd0c37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:32.140356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92e07b-36db-43c0-974c-cd7314cd0c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-bd92e07b-36db-43c0-974c-cd7314cd0c37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:32.152690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:32.153643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:32.155068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 134/256] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:32.156201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e04aa-4678-4df7-86fd-90263436909b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:32.156201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e04aa-4678-4df7-86fd-90263436909b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:32.156473 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e04aa-4678-4df7-86fd-90263436909b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:32.187993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-8a8e04aa-4678-4df7-86fd-90263436909b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:32.189200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8a8e04aa-4678-4df7-86fd-90263436909b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:32.229792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:32.268043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 135/257] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:32.268907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e04aa-4678-4df7-86fd-90263436909b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:32.300988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:32.365331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994133c-3303-4729-8526-5511e3e671d5 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:32.384032 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e04aa-4678-4df7-86fd-90263436909b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:32.451205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defc350-d0f5-4c64-a2af-4478298b084f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:32.452100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defc350-d0f5-4c64-a2af-4478298b084f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:32.452386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defc350-d0f5-4c64-a2af-4478298b084f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:32.452660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defc350-d0f5-4c64-a2af-4478298b084f None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:32.454351 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defc350-d0f5-4c64-a2af-4478298b084f None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:32.457767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defc350-d0f5-4c64-a2af-4478298b084f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:32.459726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 123/258] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:32 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:32.460342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defc350-d0f5-4c64-a2af-4478298b084f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:32.467013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df03ae-2654-43c0-8c38-26e27be5596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:32.467732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df03ae-2654-43c0-8c38-26e27be5596b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:32.468022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df03ae-2654-43c0-8c38-26e27be5596b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:32.468305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df03ae-2654-43c0-8c38-26e27be5596b None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:32.469080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df03ae-2654-43c0-8c38-26e27be5596b None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:32.469418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df03ae-2654-43c0-8c38-26e27be5596b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:32.470820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df03ae-2654-43c0-8c38-26e27be5596b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:32.488542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 136/259] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:32.489444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df03ae-2654-43c0-8c38-26e27be5596b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:32.554831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:32.656713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defc350-d0f5-4c64-a2af-4478298b084f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:32.683685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df03ae-2654-43c0-8c38-26e27be5596b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:32.698827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e763d3-8d66-4800-aed3-a0eb3d242adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:32.699653 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e763d3-8d66-4800-aed3-a0eb3d242adf None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:32.699939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e763d3-8d66-4800-aed3-a0eb3d242adf None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:32.700217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e763d3-8d66-4800-aed3-a0eb3d242adf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:32.712821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e763d3-8d66-4800-aed3-a0eb3d242adf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-89e763d3-8d66-4800-aed3-a0eb3d242adf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:32.721260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:32.722197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:32.722965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 124/260] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:32.724101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e763d3-8d66-4800-aed3-a0eb3d242adf None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:32.736720 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:32.782230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ad7d-c21d-476a-9329-22e62b6645cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:32.783200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ad7d-c21d-476a-9329-22e62b6645cf None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:32.783458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ad7d-c21d-476a-9329-22e62b6645cf None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:32.783832 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ad7d-c21d-476a-9329-22e62b6645cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:32.784806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0ad7d-c21d-476a-9329-22e62b6645cf None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=0d8860ce338747a5ba3d5903b48cc986, user.id=dc3052916c72484b8edf2b28a89a745e) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:32.785220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0ad7d-c21d-476a-9329-22e62b6645cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:32.786796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0ad7d-c21d-476a-9329-22e62b6645cf None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:32.803741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 137/261] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:32 2022] GET /identity/v3/role_assignments?user.id=dc3052916c72484b8edf2b28a89a745e&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=85420c68300642aabfd730fb8fef8347 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:32.804674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ad7d-c21d-476a-9329-22e62b6645cf None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:32.866968 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e763d3-8d66-4800-aed3-a0eb3d242adf None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:32.930280 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb69ce-e683-4fc2-901b-013decd07840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:32.931496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb69ce-e683-4fc2-901b-013decd07840 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:32.931983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb69ce-e683-4fc2-901b-013decd07840 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:32.932441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb69ce-e683-4fc2-901b-013decd07840 None admin] PATH_INFO: `/v3/users/ae8e8983b85348afb0e1c8712cbe7255` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:32.933949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb69ce-e683-4fc2-901b-013decd07840 None admin] RBAC: Authorizing `identity:get_user(user_id=ae8e8983b85348afb0e1c8712cbe7255)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:32.936183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb69ce-e683-4fc2-901b-013decd07840 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:32.938058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 125/262] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:32 2022] GET /identity/v3/users/ae8e8983b85348afb0e1c8712cbe7255 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:32.948437 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb69ce-e683-4fc2-901b-013decd07840 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:33.042750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:33.045941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:33.049800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ad7d-c21d-476a-9329-22e62b6645cf None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:33.076432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb69ce-e683-4fc2-901b-013decd07840 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:33.115862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21273799-77bd-4262-b7de-ed6356fb37d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:33.118075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21273799-77bd-4262-b7de-ed6356fb37d4 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:33.118525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21273799-77bd-4262-b7de-ed6356fb37d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:33.118955 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21273799-77bd-4262-b7de-ed6356fb37d4 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:33.120928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21273799-77bd-4262-b7de-ed6356fb37d4 None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:33.123181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21273799-77bd-4262-b7de-ed6356fb37d4 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:33.125531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 126/263] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:32 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:33.131377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21273799-77bd-4262-b7de-ed6356fb37d4 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:33.136146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538fd98-a480-46eb-a20e-8df1a1a86ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:33.137407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538fd98-a480-46eb-a20e-8df1a1a86ff0 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:33.138956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538fd98-a480-46eb-a20e-8df1a1a86ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:33.139412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538fd98-a480-46eb-a20e-8df1a1a86ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:33.141392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538fd98-a480-46eb-a20e-8df1a1a86ff0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:33.146035 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538fd98-a480-46eb-a20e-8df1a1a86ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:33.147707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538fd98-a480-46eb-a20e-8df1a1a86ff0 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:33.165495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 138/264] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:33.172888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538fd98-a480-46eb-a20e-8df1a1a86ff0 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:33.329653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538fd98-a480-46eb-a20e-8df1a1a86ff0 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:33.359765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21273799-77bd-4262-b7de-ed6356fb37d4 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:33.386069 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be32972-67bf-4035-8677-042a2417c10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:33.387398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32972-67bf-4035-8677-042a2417c10f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:33.387837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32972-67bf-4035-8677-042a2417c10f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:33.388815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32972-67bf-4035-8677-042a2417c10f None admin] PATH_INFO: `/v3/users/dc3052916c72484b8edf2b28a89a745e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:33.390389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be32972-67bf-4035-8677-042a2417c10f None admin] RBAC: Authorizing `identity:get_user(user_id=dc3052916c72484b8edf2b28a89a745e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:33.392329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be32972-67bf-4035-8677-042a2417c10f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:33.395223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 139/265] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:33 2022] GET /identity/v3/users/dc3052916c72484b8edf2b28a89a745e => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:33.400570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be32972-67bf-4035-8677-042a2417c10f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:33.457657 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:33.466859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a845c0-bb04-4eea-8493-88d2b6785d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:33.467828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a845c0-bb04-4eea-8493-88d2b6785d13 None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:33.468113 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a845c0-bb04-4eea-8493-88d2b6785d13 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:33.468395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a845c0-bb04-4eea-8493-88d2b6785d13 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986/users/ae8e8983b85348afb0e1c8712cbe7255/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:33.478251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a845c0-bb04-4eea-8493-88d2b6785d13 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8860ce338747a5ba3d5903b48cc986, user_id=ae8e8983b85348afb0e1c8712cbe7255, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:33.479744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a845c0-bb04-4eea-8493-88d2b6785d13 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:33.511834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 127/266] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:34:33 2022] PUT /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986/users/ae8e8983b85348afb0e1c8712cbe7255/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:33.521783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be32972-67bf-4035-8677-042a2417c10f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:33.567740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc10721-9946-4c5e-9e25-07205cc48691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:33.568420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc10721-9946-4c5e-9e25-07205cc48691 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:33.568711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc10721-9946-4c5e-9e25-07205cc48691 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:33.568998 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc10721-9946-4c5e-9e25-07205cc48691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:33.649273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc10721-9946-4c5e-9e25-07205cc48691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:33.650720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc10721-9946-4c5e-9e25-07205cc48691 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:33.691027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:33.767260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 140/267] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:33.772750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a845c0-bb04-4eea-8493-88d2b6785d13 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:33.786870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:33.917001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a845c0-bb04-4eea-8493-88d2b6785d13 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:33.959183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a0ddb-75a6-4ebb-bd2c-e9f9de8a01c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:33.960057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a0ddb-75a6-4ebb-bd2c-e9f9de8a01c8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:33.960343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a0ddb-75a6-4ebb-bd2c-e9f9de8a01c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:33.960685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a0ddb-75a6-4ebb-bd2c-e9f9de8a01c8 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:33.961935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a0ddb-75a6-4ebb-bd2c-e9f9de8a01c8 None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:33.963407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a0ddb-75a6-4ebb-bd2c-e9f9de8a01c8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:33.965617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 128/268] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:33 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:33.972151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc10721-9946-4c5e-9e25-07205cc48691 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:34.041901 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36952e50-2ac2-42a6-9ea6-a6b677cb1706 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.042239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36952e50-2ac2-42a6-9ea6-a6b677cb1706 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.042468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36952e50-2ac2-42a6-9ea6-a6b677cb1706 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.043924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 129/269] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:34:34.054185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713dc7b-312f-4efe-b944-17d0f88512fa None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.054476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713dc7b-312f-4efe-b944-17d0f88512fa None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.054737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713dc7b-312f-4efe-b944-17d0f88512fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.083824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a713dc7b-312f-4efe-b944-17d0f88512fa None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:34.085128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a713dc7b-312f-4efe-b944-17d0f88512fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:34.100692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc10721-9946-4c5e-9e25-07205cc48691 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:34.154434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:34.173125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c1520-95d4-4e7f-ad73-fe9e274dec89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:34.174008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c1520-95d4-4e7f-ad73-fe9e274dec89 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.174662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c1520-95d4-4e7f-ad73-fe9e274dec89 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.175148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c1520-95d4-4e7f-ad73-fe9e274dec89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.176901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c1520-95d4-4e7f-ad73-fe9e274dec89 None admin] RBAC: Adding query filter params (role.id=0044294fcceb4b6ba0cb27567f2761d4, scope.project.id=0d8860ce338747a5ba3d5903b48cc986, user.id=dc3052916c72484b8edf2b28a89a745e) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:34.177265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c1520-95d4-4e7f-ad73-fe9e274dec89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:34.179177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c1520-95d4-4e7f-ad73-fe9e274dec89 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:34.194931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 141/270] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:33 2022] GET /identity/v3/role_assignments?user.id=dc3052916c72484b8edf2b28a89a745e&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=0044294fcceb4b6ba0cb27567f2761d4 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:34.196276 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc7ca6-48e7-4466-ae56-b89e7eb601e1 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.196563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc7ca6-48e7-4466-ae56-b89e7eb601e1 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.196856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc7ca6-48e7-4466-ae56-b89e7eb601e1 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.206800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 142/271] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:34 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:34.212187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 130/272] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:34.213455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb46ac-b642-4ba2-928f-f47c35b0e335 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.213753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb46ac-b642-4ba2-928f-f47c35b0e335 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.214128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb46ac-b642-4ba2-928f-f47c35b0e335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.244252 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-94eb46ac-b642-4ba2-928f-f47c35b0e335 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:34.244992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713dc7b-312f-4efe-b944-17d0f88512fa None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:34.245532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-94eb46ac-b642-4ba2-928f-f47c35b0e335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:34.249238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:34.357903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 143/273] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:34.417244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb46ac-b642-4ba2-928f-f47c35b0e335 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:34.429696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713dc7b-312f-4efe-b944-17d0f88512fa None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:34.481405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af7b15-c907-4f79-bc34-4ffc483d34e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:34.483746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af7b15-c907-4f79-bc34-4ffc483d34e8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.483746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af7b15-c907-4f79-bc34-4ffc483d34e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.483946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af7b15-c907-4f79-bc34-4ffc483d34e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.498894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af7b15-c907-4f79-bc34-4ffc483d34e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-07af7b15-c907-4f79-bc34-4ffc483d34e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:34.508597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:34.519005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:34.519818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 131/274] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:34:34.523950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af7b15-c907-4f79-bc34-4ffc483d34e8 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:34.570213 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb46ac-b642-4ba2-928f-f47c35b0e335 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:34.603400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:34.607563 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:34.634350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc50e0-1e58-4db2-a534-005f5a5ddf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:34.635205 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc50e0-1e58-4db2-a534-005f5a5ddf82 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.635457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc50e0-1e58-4db2-a534-005f5a5ddf82 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.635815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc50e0-1e58-4db2-a534-005f5a5ddf82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.637129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc50e0-1e58-4db2-a534-005f5a5ddf82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:34.638687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc50e0-1e58-4db2-a534-005f5a5ddf82 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:34.640377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 144/275] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:34.646023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc50e0-1e58-4db2-a534-005f5a5ddf82 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:34.658905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af7b15-c907-4f79-bc34-4ffc483d34e8 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:34.730591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f31ff-a4b9-4c41-b757-a2c50af1fc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:34.732366 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f31ff-a4b9-4c41-b757-a2c50af1fc3b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.732807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f31ff-a4b9-4c41-b757-a2c50af1fc3b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.733232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f31ff-a4b9-4c41-b757-a2c50af1fc3b None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.734366 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f31ff-a4b9-4c41-b757-a2c50af1fc3b None admin] RBAC: Adding query filter params (name=member) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:34.734827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f31ff-a4b9-4c41-b757-a2c50af1fc3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:34.737054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f31ff-a4b9-4c41-b757-a2c50af1fc3b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:34.748644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 132/276] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:34.750244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85925c4-1fb6-4a93-a47f-ecf552c43549 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.750602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85925c4-1fb6-4a93-a47f-ecf552c43549 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.751051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85925c4-1fb6-4a93-a47f-ecf552c43549 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.752647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 133/277] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:34.756875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85925c4-1fb6-4a93-a47f-ecf552c43549 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:34.814186 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc50e0-1e58-4db2-a534-005f5a5ddf82 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:34.873587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b66932-a677-4730-baf6-e53dbf570553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:34.874214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b66932-a677-4730-baf6-e53dbf570553 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.874505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b66932-a677-4730-baf6-e53dbf570553 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.874781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b66932-a677-4730-baf6-e53dbf570553 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.875561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b66932-a677-4730-baf6-e53dbf570553 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:34.877167 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b66932-a677-4730-baf6-e53dbf570553 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:34.892612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85925c4-1fb6-4a93-a47f-ecf552c43549 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:34.905895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 145/278] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:34:34 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:34:34.915231 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b30de-30c2-4b2e-b4ee-955099d95175 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.915554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b30de-30c2-4b2e-b4ee-955099d95175 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.915799 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b30de-30c2-4b2e-b4ee-955099d95175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.950234 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-319b30de-30c2-4b2e-b4ee-955099d95175 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:34.951033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb08ef-2bae-4d6f-a45a-ee92919d99a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:34.952301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-319b30de-30c2-4b2e-b4ee-955099d95175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:34.952513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb08ef-2bae-4d6f-a45a-ee92919d99a1 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:34.952976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb08ef-2bae-4d6f-a45a-ee92919d99a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:34.953474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb08ef-2bae-4d6f-a45a-ee92919d99a1 None admin] PATH_INFO: `/v3/users/b571ffab89b3479687aa4fed351fa667` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:34.960361 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:34.979010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb08ef-2bae-4d6f-a45a-ee92919d99a1 None admin] RBAC: Authorizing `identity:get_user(user_id=b571ffab89b3479687aa4fed351fa667)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:34.980583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb08ef-2bae-4d6f-a45a-ee92919d99a1 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:34.989531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 134/279] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:34 2022] GET /identity/v3/users/b571ffab89b3479687aa4fed351fa667 => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:34.994121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb08ef-2bae-4d6f-a45a-ee92919d99a1 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:35.031745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:35.072952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 146/280] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:35.097328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b30de-30c2-4b2e-b4ee-955099d95175 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:35.146615 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb08ef-2bae-4d6f-a45a-ee92919d99a1 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:35.197819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9b23c-154c-42ce-96a7-f6185c0100f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:35.199195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9b23c-154c-42ce-96a7-f6185c0100f0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:35.199487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9b23c-154c-42ce-96a7-f6185c0100f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:35.199770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9b23c-154c-42ce-96a7-f6185c0100f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:35.201151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9b23c-154c-42ce-96a7-f6185c0100f0 None admin] RBAC: Adding query filter params (role.id=d94de411ec0a40be92c04950e6edcac7, user.id=b571ffab89b3479687aa4fed351fa667, scope.system=all) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:35.202475 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9b23c-154c-42ce-96a7-f6185c0100f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:35.203967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9b23c-154c-42ce-96a7-f6185c0100f0 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:35.209950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 135/281] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Jul 19 20:34:34 2022] GET /identity/v3/role_assignments?user.id=b571ffab89b3479687aa4fed351fa667&scope.system=all&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:35.221318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b30de-30c2-4b2e-b4ee-955099d95175 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:35.268881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574216dc-4b8a-4073-a41c-cc55d2570c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:35.269661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574216dc-4b8a-4073-a41c-cc55d2570c3f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:35.269948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574216dc-4b8a-4073-a41c-cc55d2570c3f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:35.270231 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574216dc-4b8a-4073-a41c-cc55d2570c3f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:35.284427 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574216dc-4b8a-4073-a41c-cc55d2570c3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-574216dc-4b8a-4073-a41c-cc55d2570c3f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:35.292770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:35.293764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:35.294760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 147/282] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:34:35.300369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9b23c-154c-42ce-96a7-f6185c0100f0 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:35.373899 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:35.448774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9b23c-154c-42ce-96a7-f6185c0100f0 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:35.507485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c1459-0a4d-4f35-b8c2-c03ca5c5c600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:35.508281 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1459-0a4d-4f35-b8c2-c03ca5c5c600 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:35.508622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1459-0a4d-4f35-b8c2-c03ca5c5c600 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:35.508902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c1459-0a4d-4f35-b8c2-c03ca5c5c600 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:35.509764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c1459-0a4d-4f35-b8c2-c03ca5c5c600 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:35.510076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c1459-0a4d-4f35-b8c2-c03ca5c5c600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:35.511479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c1459-0a4d-4f35-b8c2-c03ca5c5c600 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:35.526614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 136/283] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:35.534527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574216dc-4b8a-4073-a41c-cc55d2570c3f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:35.576847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:35.672238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574216dc-4b8a-4073-a41c-cc55d2570c3f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:35.716856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0371bf-b401-4c33-b48a-a81d6d2ad1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:35.717768 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0371bf-b401-4c33-b48a-a81d6d2ad1f0 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:35.718056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0371bf-b401-4c33-b48a-a81d6d2ad1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:35.718334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0371bf-b401-4c33-b48a-a81d6d2ad1f0 None admin] PATH_INFO: `/v3/users/1250f110be6f46e990a0cb4ae26a141c` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:35.719564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0371bf-b401-4c33-b48a-a81d6d2ad1f0 None admin] RBAC: Authorizing `identity:get_user(user_id=1250f110be6f46e990a0cb4ae26a141c)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:35.721020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0371bf-b401-4c33-b48a-a81d6d2ad1f0 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:35.722704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 148/284] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:35 2022] GET /identity/v3/users/1250f110be6f46e990a0cb4ae26a141c => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:35.728391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c1459-0a4d-4f35-b8c2-c03ca5c5c600 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:35.879404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c1459-0a4d-4f35-b8c2-c03ca5c5c600 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:35.890418 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:35.916195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16521df7-2ac7-4a13-b969-3c1cbe53e2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:35.917067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16521df7-2ac7-4a13-b969-3c1cbe53e2ab None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:35.917370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16521df7-2ac7-4a13-b969-3c1cbe53e2ab None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:35.917661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16521df7-2ac7-4a13-b969-3c1cbe53e2ab None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:35.918887 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16521df7-2ac7-4a13-b969-3c1cbe53e2ab None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:35.920895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16521df7-2ac7-4a13-b969-3c1cbe53e2ab None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:35.922997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 137/285] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:35 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:35.928556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0371bf-b401-4c33-b48a-a81d6d2ad1f0 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:36.093673 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0371bf-b401-4c33-b48a-a81d6d2ad1f0 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:36.102023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:36.148146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7807822-14aa-4d20-b851-4b83be612508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:36.148972 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7807822-14aa-4d20-b851-4b83be612508 None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:36.149261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7807822-14aa-4d20-b851-4b83be612508 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:36.149568 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7807822-14aa-4d20-b851-4b83be612508 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986/users/1250f110be6f46e990a0cb4ae26a141c/roles/74ea4fc9cad8496a831bb325feb84d4c` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:36.151627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7807822-14aa-4d20-b851-4b83be612508 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8860ce338747a5ba3d5903b48cc986, user_id=1250f110be6f46e990a0cb4ae26a141c, role_id=74ea4fc9cad8496a831bb325feb84d4c)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:36.153070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7807822-14aa-4d20-b851-4b83be612508 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:36.180309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 149/286] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:34:35 2022] PUT /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986/users/1250f110be6f46e990a0cb4ae26a141c/roles/74ea4fc9cad8496a831bb325feb84d4c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:36.211140 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf62b0-674e-472e-b2a5-65376d689fe2 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:36.211452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf62b0-674e-472e-b2a5-65376d689fe2 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:36.211707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf62b0-674e-472e-b2a5-65376d689fe2 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:36.213522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 138/287] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:36.219856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f5181-1dd3-4b32-855b-eaa134d560dc None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:36.220142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f5181-1dd3-4b32-855b-eaa134d560dc None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:36.220432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f5181-1dd3-4b32-855b-eaa134d560dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:36.250662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0e6f5181-1dd3-4b32-855b-eaa134d560dc None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:36.251921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0e6f5181-1dd3-4b32-855b-eaa134d560dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:36.344650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 150/288] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:36.386928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf62b0-674e-472e-b2a5-65376d689fe2 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:36.518194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf62b0-674e-472e-b2a5-65376d689fe2 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:36.554455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ad210-e674-4f63-82e6-41336b9141f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:36.555654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ad210-e674-4f63-82e6-41336b9141f1 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:36.556160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ad210-e674-4f63-82e6-41336b9141f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:36.556628 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ad210-e674-4f63-82e6-41336b9141f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:36.568210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ad210-e674-4f63-82e6-41336b9141f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-9b9ad210-e674-4f63-82e6-41336b9141f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:36.577067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:36.577964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:36.578858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:36.581596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 139/289] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:36.587263 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f5181-1dd3-4b32-855b-eaa134d560dc None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:36.715260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:36.731688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f5181-1dd3-4b32-855b-eaa134d560dc None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:36.781660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8877387-b8a6-4d68-805a-f0f533c32b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:36.782273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8877387-b8a6-4d68-805a-f0f533c32b7f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:36.782564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8877387-b8a6-4d68-805a-f0f533c32b7f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:36.782846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8877387-b8a6-4d68-805a-f0f533c32b7f None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:36.783614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8877387-b8a6-4d68-805a-f0f533c32b7f None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:36.783926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8877387-b8a6-4d68-805a-f0f533c32b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:36.785312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8877387-b8a6-4d68-805a-f0f533c32b7f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:36.794706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 151/290] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:36.800315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ad210-e674-4f63-82e6-41336b9141f1 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:36.871928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:36.958079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ad210-e674-4f63-82e6-41336b9141f1 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:37.014897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8dd5b1-5cb8-461e-866d-d8a0fccb4ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:37.015780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dd5b1-5cb8-461e-866d-d8a0fccb4ce4 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:37.016062 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dd5b1-5cb8-461e-866d-d8a0fccb4ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:37.016336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dd5b1-5cb8-461e-866d-d8a0fccb4ce4 None admin] PATH_INFO: `/v3/users/ae8e8983b85348afb0e1c8712cbe7255` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:37.022168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8dd5b1-5cb8-461e-866d-d8a0fccb4ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=ae8e8983b85348afb0e1c8712cbe7255)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:37.023759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8dd5b1-5cb8-461e-866d-d8a0fccb4ce4 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:37.025899 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 140/291] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:36 2022] GET /identity/v3/users/ae8e8983b85348afb0e1c8712cbe7255 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:37.031566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8877387-b8a6-4d68-805a-f0f533c32b7f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:37.215344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8877387-b8a6-4d68-805a-f0f533c32b7f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:37.264114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f1204-ee46-4b70-b0a4-70c7ea12fd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:37.265630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1204-ee46-4b70-b0a4-70c7ea12fd45 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:37.266180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1204-ee46-4b70-b0a4-70c7ea12fd45 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:37.266664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1204-ee46-4b70-b0a4-70c7ea12fd45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:37.307692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:37.370325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f1204-ee46-4b70-b0a4-70c7ea12fd45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:37.370507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f1204-ee46-4b70-b0a4-70c7ea12fd45 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:37.518363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 152/292] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:37.526079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8dd5b1-5cb8-461e-866d-d8a0fccb4ce4 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:37.709185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b880bc-49c3-4b77-a6a8-f516458e671e None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:37.709524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b880bc-49c3-4b77-a6a8-f516458e671e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:37.709804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b880bc-49c3-4b77-a6a8-f516458e671e None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:37.711341 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 153/293] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:37.716969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57835879-30bf-400e-bd33-b38a5ec952df None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:37.717263 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57835879-30bf-400e-bd33-b38a5ec952df None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:37.717591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57835879-30bf-400e-bd33-b38a5ec952df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:37.724668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8dd5b1-5cb8-461e-866d-d8a0fccb4ce4 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:37.738087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:37.755534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-57835879-30bf-400e-bd33-b38a5ec952df None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:37.756881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-57835879-30bf-400e-bd33-b38a5ec952df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:37.805442 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fe794-0bde-4b8c-a82f-8905ec593d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:37.806322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fe794-0bde-4b8c-a82f-8905ec593d77 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:37.806598 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fe794-0bde-4b8c-a82f-8905ec593d77 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:37.806868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fe794-0bde-4b8c-a82f-8905ec593d77 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:37.808356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fe794-0bde-4b8c-a82f-8905ec593d77 None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:37.809884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fe794-0bde-4b8c-a82f-8905ec593d77 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:37.812123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 141/294] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:37 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:37.821172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fe794-0bde-4b8c-a82f-8905ec593d77 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:37.855401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:37.877629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 154/295] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:37.878881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257a09-40c3-4e56-b3ef-fd7bbc0287d4 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:37.879166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257a09-40c3-4e56-b3ef-fd7bbc0287d4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:37.879441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d257a09-40c3-4e56-b3ef-fd7bbc0287d4 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:37.880903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 155/296] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:37.898954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82ae4-ce5d-4e68-835d-2ec1605f77fc None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:37.899260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82ae4-ce5d-4e68-835d-2ec1605f77fc None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:37.899506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82ae4-ce5d-4e68-835d-2ec1605f77fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:37.934178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-71a82ae4-ce5d-4e68-835d-2ec1605f77fc None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:37.937658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-71a82ae4-ce5d-4e68-835d-2ec1605f77fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:37.982575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fe794-0bde-4b8c-a82f-8905ec593d77 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:38.024477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c936b7-9c5c-4b10-9c4e-d5625b91b48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:38.025634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c936b7-9c5c-4b10-9c4e-d5625b91b48c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.026126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c936b7-9c5c-4b10-9c4e-d5625b91b48c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.026634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c936b7-9c5c-4b10-9c4e-d5625b91b48c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.027706 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c936b7-9c5c-4b10-9c4e-d5625b91b48c None admin] RBAC: Adding query filter params (role.id=d94de411ec0a40be92c04950e6edcac7, scope.project.id=0d8860ce338747a5ba3d5903b48cc986, user.id=ae8e8983b85348afb0e1c8712cbe7255) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:38.028242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c936b7-9c5c-4b10-9c4e-d5625b91b48c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:38.030577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c936b7-9c5c-4b10-9c4e-d5625b91b48c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:38.039692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 142/297] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:37 2022] GET /identity/v3/role_assignments?user.id=ae8e8983b85348afb0e1c8712cbe7255&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:38.041217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c936b7-9c5c-4b10-9c4e-d5625b91b48c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:38.093506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 156/298] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:38.114011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:38.144742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a82ae4-ce5d-4e68-835d-2ec1605f77fc None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:38.236756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c936b7-9c5c-4b10-9c4e-d5625b91b48c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:38.291794 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52bad9c-2bbf-454d-924b-4507fec3b507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:38.292607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bad9c-2bbf-454d-924b-4507fec3b507 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.292891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bad9c-2bbf-454d-924b-4507fec3b507 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.293167 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bad9c-2bbf-454d-924b-4507fec3b507 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.315376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a82ae4-ce5d-4e68-835d-2ec1605f77fc None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:38.316577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52bad9c-2bbf-454d-924b-4507fec3b507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-d52bad9c-2bbf-454d-924b-4507fec3b507 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:38.324779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:38.326614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 143/299] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:38.328603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a669c4c-b9dc-4c62-9197-eb00c112aa28 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.329581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a669c4c-b9dc-4c62-9197-eb00c112aa28 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.329581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a669c4c-b9dc-4c62-9197-eb00c112aa28 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.330065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 144/300] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:38.337010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:38.339318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a669c4c-b9dc-4c62-9197-eb00c112aa28 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:38.364846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55cf25-ec94-457f-b16c-e93383a5965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:38.365651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55cf25-ec94-457f-b16c-e93383a5965e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.365934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55cf25-ec94-457f-b16c-e93383a5965e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.366210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55cf25-ec94-457f-b16c-e93383a5965e None admin] PATH_INFO: `/v3/domains/default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.367541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55cf25-ec94-457f-b16c-e93383a5965e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:38.369004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55cf25-ec94-457f-b16c-e93383a5965e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:38.371795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 157/301] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:38.373149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eff3b-80d4-4726-8ce8-ffc7da221756 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.373620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eff3b-80d4-4726-8ce8-ffc7da221756 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.373904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018eff3b-80d4-4726-8ce8-ffc7da221756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.394218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-018eff3b-80d4-4726-8ce8-ffc7da221756 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:38.395249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-018eff3b-80d4-4726-8ce8-ffc7da221756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:38.461986 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:38.467915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a669c4c-b9dc-4c62-9197-eb00c112aa28 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:38.470957 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 158/302] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:38.471788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018eff3b-80d4-4726-8ce8-ffc7da221756 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:38.518077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa1631-4a1f-48c1-9ae3-f9a70ffb53f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:38.518734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa1631-4a1f-48c1-9ae3-f9a70ffb53f9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.519017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa1631-4a1f-48c1-9ae3-f9a70ffb53f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.519295 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa1631-4a1f-48c1-9ae3-f9a70ffb53f9 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.520042 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa1631-4a1f-48c1-9ae3-f9a70ffb53f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:38.520348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa1631-4a1f-48c1-9ae3-f9a70ffb53f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:38.521757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa1631-4a1f-48c1-9ae3-f9a70ffb53f9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:38.530888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 145/303] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:38.536567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa1631-4a1f-48c1-9ae3-f9a70ffb53f9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:38.595652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018eff3b-80d4-4726-8ce8-ffc7da221756 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:38.610570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:38.635639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26456d58-356d-4bf2-acc5-52a4ae6d857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:38.636230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26456d58-356d-4bf2-acc5-52a4ae6d857b None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.636520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26456d58-356d-4bf2-acc5-52a4ae6d857b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.636798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26456d58-356d-4bf2-acc5-52a4ae6d857b None admin] PATH_INFO: `/v3/groups` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.637526 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26456d58-356d-4bf2-acc5-52a4ae6d857b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:38.639596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26456d58-356d-4bf2-acc5-52a4ae6d857b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:38.652660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa1631-4a1f-48c1-9ae3-f9a70ffb53f9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:38.654660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 159/304] 10.222.0.70 () {60 vars in 1134 bytes} [Tue Jul 19 20:34:38 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:34:38.655458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26456d58-356d-4bf2-acc5-52a4ae6d857b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:38.722902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043df10-569c-4cc4-892a-81db2058b51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:38.726830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043df10-569c-4cc4-892a-81db2058b51f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.728274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043df10-569c-4cc4-892a-81db2058b51f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.729703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043df10-569c-4cc4-892a-81db2058b51f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.750829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c043df10-569c-4cc4-892a-81db2058b51f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-c043df10-569c-4cc4-892a-81db2058b51f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:38.760941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:38.761940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:38.762726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:38.768603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 146/305] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:38.775202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043df10-569c-4cc4-892a-81db2058b51f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:38.800752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26456d58-356d-4bf2-acc5-52a4ae6d857b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:38.807012 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:38.842977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25daec-561b-479a-8ff1-b30ad7edbe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:38.843989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25daec-561b-479a-8ff1-b30ad7edbe46 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.844273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25daec-561b-479a-8ff1-b30ad7edbe46 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.844554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25daec-561b-479a-8ff1-b30ad7edbe46 None admin] PATH_INFO: `/v3/users/259572ae4cc141df9e8bdc4ebc9647eb` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.867778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25daec-561b-479a-8ff1-b30ad7edbe46 None admin] RBAC: Authorizing `identity:get_user(user_id=259572ae4cc141df9e8bdc4ebc9647eb)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:38.869890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25daec-561b-479a-8ff1-b30ad7edbe46 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:38.873124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 160/306] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:38 2022] GET /identity/v3/users/259572ae4cc141df9e8bdc4ebc9647eb => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:38.874720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad741c60-826e-462a-b3b2-6049cabe7b84 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.875194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad741c60-826e-462a-b3b2-6049cabe7b84 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.875742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad741c60-826e-462a-b3b2-6049cabe7b84 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.877319 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 161/307] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:38.880660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad741c60-826e-462a-b3b2-6049cabe7b84 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:38.904735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043df10-569c-4cc4-892a-81db2058b51f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:38.924180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:38.945898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654bbe8-a993-4b28-bcd9-bb6b3af96b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:38.946971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654bbe8-a993-4b28-bcd9-bb6b3af96b21 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.947426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654bbe8-a993-4b28-bcd9-bb6b3af96b21 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.947871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654bbe8-a993-4b28-bcd9-bb6b3af96b21 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.949025 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654bbe8-a993-4b28-bcd9-bb6b3af96b21 None admin] RBAC: Adding query filter params (name=member) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:38.949518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654bbe8-a993-4b28-bcd9-bb6b3af96b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:38.951102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654bbe8-a993-4b28-bcd9-bb6b3af96b21 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:38.960146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 147/308] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:38.960146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c912cb6-f519-4bb5-87bf-01711a8a859d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:38.960146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c912cb6-f519-4bb5-87bf-01711a8a859d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:38.960146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c912cb6-f519-4bb5-87bf-01711a8a859d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:38.982251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0c912cb6-f519-4bb5-87bf-01711a8a859d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:38.983471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0c912cb6-f519-4bb5-87bf-01711a8a859d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:39.011247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad741c60-826e-462a-b3b2-6049cabe7b84 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:39.050007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dd7fc-b758-46e1-825f-8bab040f97a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:39.050968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dd7fc-b758-46e1-825f-8bab040f97a8 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:39.051415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dd7fc-b758-46e1-825f-8bab040f97a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:39.051831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dd7fc-b758-46e1-825f-8bab040f97a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:39.052822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dd7fc-b758-46e1-825f-8bab040f97a8 None admin] RBAC: Adding query filter params (role.id=74ea4fc9cad8496a831bb325feb84d4c, user.id=259572ae4cc141df9e8bdc4ebc9647eb, scope.system=all) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:39.053273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dd7fc-b758-46e1-825f-8bab040f97a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:39.054888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dd7fc-b758-46e1-825f-8bab040f97a8 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:39.056431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 148/309] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:39.057273 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c912cb6-f519-4bb5-87bf-01711a8a859d None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:39.066685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 162/310] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Jul 19 20:34:38 2022] GET /identity/v3/role_assignments?user.id=259572ae4cc141df9e8bdc4ebc9647eb&scope.system=all&role.id=74ea4fc9cad8496a831bb325feb84d4c => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:39.089098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dd7fc-b758-46e1-825f-8bab040f97a8 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:39.091799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:39.216907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:39.225643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dd7fc-b758-46e1-825f-8bab040f97a8 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:39.229343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c912cb6-f519-4bb5-87bf-01711a8a859d None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:39.271048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ae4c4-f3bd-462c-b2e3-660dc94f2942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:39.271870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ae4c4-f3bd-462c-b2e3-660dc94f2942 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:39.272154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ae4c4-f3bd-462c-b2e3-660dc94f2942 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:39.272426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ae4c4-f3bd-462c-b2e3-660dc94f2942 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:39.272718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7d2fe-7575-4190-a064-d30d06705b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:39.273656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7d2fe-7575-4190-a064-d30d06705b82 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:39.273942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7d2fe-7575-4190-a064-d30d06705b82 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:39.274219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7d2fe-7575-4190-a064-d30d06705b82 None admin] PATH_INFO: `/v3/users/b571ffab89b3479687aa4fed351fa667` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:39.275462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7d2fe-7575-4190-a064-d30d06705b82 None admin] RBAC: Authorizing `identity:get_user(user_id=b571ffab89b3479687aa4fed351fa667)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:39.276924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7d2fe-7575-4190-a064-d30d06705b82 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:39.278912 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 149/311] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:39 2022] GET /identity/v3/users/b571ffab89b3479687aa4fed351fa667 => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:39.284783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7d2fe-7575-4190-a064-d30d06705b82 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:39.285718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ae4c4-f3bd-462c-b2e3-660dc94f2942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-bc0ae4c4-f3bd-462c-b2e3-660dc94f2942 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:39.293717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:39.294845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:39.295620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 163/312] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:39.300681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ae4c4-f3bd-462c-b2e3-660dc94f2942 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:39.421842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:39.423343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ae4c4-f3bd-462c-b2e3-660dc94f2942 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:39.443425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7d2fe-7575-4190-a064-d30d06705b82 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:39.466471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046795d-7d72-41cb-8e99-a3eb6dfc5b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:39.468033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046795d-7d72-41cb-8e99-a3eb6dfc5b70 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:39.468789 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046795d-7d72-41cb-8e99-a3eb6dfc5b70 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:39.469380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046795d-7d72-41cb-8e99-a3eb6dfc5b70 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:39.470878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046795d-7d72-41cb-8e99-a3eb6dfc5b70 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:39.471542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046795d-7d72-41cb-8e99-a3eb6dfc5b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:39.473944 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046795d-7d72-41cb-8e99-a3eb6dfc5b70 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:39.481161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7e79c-5abb-434b-875e-0ef9c4891780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:39.482236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7e79c-5abb-434b-875e-0ef9c4891780 None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:39.482518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7e79c-5abb-434b-875e-0ef9c4891780 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:39.482795 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7e79c-5abb-434b-875e-0ef9c4891780 None admin] PATH_INFO: `/v3/system/users/b571ffab89b3479687aa4fed351fa667/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:39.484339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7e79c-5abb-434b-875e-0ef9c4891780 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b571ffab89b3479687aa4fed351fa667, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:39.485767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7e79c-5abb-434b-875e-0ef9c4891780 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:39.486214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 164/313] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:39.491844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046795d-7d72-41cb-8e99-a3eb6dfc5b70 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:39.498410 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 150/314] 10.222.0.70 () {58 vars in 1334 bytes} [Tue Jul 19 20:34:39 2022] PUT /identity/v3/system/users/b571ffab89b3479687aa4fed351fa667/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:34:39.517492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:39.625302 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046795d-7d72-41cb-8e99-a3eb6dfc5b70 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:39.664720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0515de-af61-4cd6-b64e-dcbd17f3f00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:39.665637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0515de-af61-4cd6-b64e-dcbd17f3f00e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:39.665924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0515de-af61-4cd6-b64e-dcbd17f3f00e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:39.666199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0515de-af61-4cd6-b64e-dcbd17f3f00e None admin] PATH_INFO: `/v3/users/1250f110be6f46e990a0cb4ae26a141c` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:39.667546 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0515de-af61-4cd6-b64e-dcbd17f3f00e None admin] RBAC: Authorizing `identity:get_user(user_id=1250f110be6f46e990a0cb4ae26a141c)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:39.668981 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0515de-af61-4cd6-b64e-dcbd17f3f00e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:39.670941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 165/315] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:39 2022] GET /identity/v3/users/1250f110be6f46e990a0cb4ae26a141c => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:39.676993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7e79c-5abb-434b-875e-0ef9c4891780 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:39.790061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7e79c-5abb-434b-875e-0ef9c4891780 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:39.825822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b89ef-7069-419c-b08c-956d8275b747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:39.826790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b89ef-7069-419c-b08c-956d8275b747 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:39.827253 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b89ef-7069-419c-b08c-956d8275b747 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:39.827704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b89ef-7069-419c-b08c-956d8275b747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:39.851249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:39.904225 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b89ef-7069-419c-b08c-956d8275b747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:39.905725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b89ef-7069-419c-b08c-956d8275b747 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:40.011126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 151/316] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:34:40.016668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0515de-af61-4cd6-b64e-dcbd17f3f00e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:40.107786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:40.123871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0515de-af61-4cd6-b64e-dcbd17f3f00e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:40.162902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c1f37-ea93-4760-adf0-088e33926854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:40.163774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c1f37-ea93-4760-adf0-088e33926854 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:40.164062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c1f37-ea93-4760-adf0-088e33926854 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:40.164342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c1f37-ea93-4760-adf0-088e33926854 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:40.165581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c1f37-ea93-4760-adf0-088e33926854 None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:40.167013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c1f37-ea93-4760-adf0-088e33926854 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:40.169121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 166/317] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:40 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:40.176796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b89ef-7069-419c-b08c-956d8275b747 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:40.225565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:40.283285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b89ef-7069-419c-b08c-956d8275b747 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:40.321540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18648f-f83d-4b55-a9e0-f8bfeb7182db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:40.322665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18648f-f83d-4b55-a9e0-f8bfeb7182db None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:40.322950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18648f-f83d-4b55-a9e0-f8bfeb7182db None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:40.323230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18648f-f83d-4b55-a9e0-f8bfeb7182db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:40.324326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18648f-f83d-4b55-a9e0-f8bfeb7182db None admin] RBAC: Adding query filter params (role.id=74ea4fc9cad8496a831bb325feb84d4c, scope.project.id=0d8860ce338747a5ba3d5903b48cc986, user.id=1250f110be6f46e990a0cb4ae26a141c) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:40.324640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18648f-f83d-4b55-a9e0-f8bfeb7182db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:40.326150 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18648f-f83d-4b55-a9e0-f8bfeb7182db None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:40.333017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 152/318] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:34:40 2022] GET /identity/v3/role_assignments?user.id=1250f110be6f46e990a0cb4ae26a141c&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=74ea4fc9cad8496a831bb325feb84d4c => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:40.884408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9420b-c38f-4f1a-abfe-b228d26b4a24 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:40.884929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9420b-c38f-4f1a-abfe-b228d26b4a24 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:40.884929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9420b-c38f-4f1a-abfe-b228d26b4a24 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:40.886342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 167/319] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:40.892019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc471f45-0e24-475f-a559-e53d6eff9de6 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:40.892507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc471f45-0e24-475f-a559-e53d6eff9de6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:40.892660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc471f45-0e24-475f-a559-e53d6eff9de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:40.909486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-bc471f45-0e24-475f-a559-e53d6eff9de6 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:40.910668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bc471f45-0e24-475f-a559-e53d6eff9de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:40.977166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 153/320] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:41.000065 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9420b-c38f-4f1a-abfe-b228d26b4a24 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:41.105153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9420b-c38f-4f1a-abfe-b228d26b4a24 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:41.139874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70169dbf-8a5c-4e4a-ac3f-7e634620d8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:41.140594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169dbf-8a5c-4e4a-ac3f-7e634620d8af None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:41.140867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169dbf-8a5c-4e4a-ac3f-7e634620d8af None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:41.141144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169dbf-8a5c-4e4a-ac3f-7e634620d8af None admin] PATH_INFO: `/v3/domains/default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:41.142336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70169dbf-8a5c-4e4a-ac3f-7e634620d8af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:41.143790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70169dbf-8a5c-4e4a-ac3f-7e634620d8af None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:41.145922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 168/321] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:34:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:41.151478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc471f45-0e24-475f-a559-e53d6eff9de6 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:41.200812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:41.271891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6672-5282-4a09-ade4-2ed58d08459d None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:41.272216 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6672-5282-4a09-ade4-2ed58d08459d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:41.272499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6672-5282-4a09-ade4-2ed58d08459d None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:41.274131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 169/322] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:41.279508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b784fe-585a-493b-b9c7-78b726f3f79f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:41.279508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b784fe-585a-493b-b9c7-78b726f3f79f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:41.279812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b784fe-585a-493b-b9c7-78b726f3f79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:41.286136 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc471f45-0e24-475f-a559-e53d6eff9de6 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:41.297192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-69b784fe-585a-493b-b9c7-78b726f3f79f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:41.298436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-69b784fe-585a-493b-b9c7-78b726f3f79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:41.314077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:41.323203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9528d76-e8c4-49ae-830e-ca83b580b34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:41.324127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9528d76-e8c4-49ae-830e-ca83b580b34e None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:41.324410 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9528d76-e8c4-49ae-830e-ca83b580b34e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:41.324682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9528d76-e8c4-49ae-830e-ca83b580b34e None admin] PATH_INFO: `/v3/groups` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:41.325429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9528d76-e8c4-49ae-830e-ca83b580b34e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:41.326868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9528d76-e8c4-49ae-830e-ca83b580b34e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:41.349407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 154/323] 10.222.0.70 () {60 vars in 1134 bytes} [Tue Jul 19 20:34:41 2022] POST /identity/v3/groups => generated 226 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:34:41.384498 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 170/324] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:41.408546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9528d76-e8c4-49ae-830e-ca83b580b34e None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:41.517438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:41.521464 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9528d76-e8c4-49ae-830e-ca83b580b34e None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:41.562630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af130bc3-c3c1-4874-805e-dd290bba7da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:41.563426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af130bc3-c3c1-4874-805e-dd290bba7da2 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:41.563716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af130bc3-c3c1-4874-805e-dd290bba7da2 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:41.563992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af130bc3-c3c1-4874-805e-dd290bba7da2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:41.575928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af130bc3-c3c1-4874-805e-dd290bba7da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-af130bc3-c3c1-4874-805e-dd290bba7da2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:41.584029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:41.584987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:41.586163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 155/325] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:41.595809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b784fe-585a-493b-b9c7-78b726f3f79f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:41.699404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056f697-0109-4529-8f71-e1c5d8c8c5fa None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:41.699984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056f697-0109-4529-8f71-e1c5d8c8c5fa None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:41.700510 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056f697-0109-4529-8f71-e1c5d8c8c5fa None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:41.702282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 156/326] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:41.707855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93306-9aec-4d89-a445-715f0ef270e0 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:41.708435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93306-9aec-4d89-a445-715f0ef270e0 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:41.708955 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93306-9aec-4d89-a445-715f0ef270e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:41.720727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b784fe-585a-493b-b9c7-78b726f3f79f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:41.730717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-7ae93306-9aec-4d89-a445-715f0ef270e0 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:41.732496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7ae93306-9aec-4d89-a445-715f0ef270e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:41.765996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84dd89-d89d-43bd-abb2-904413e5001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:41.767533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84dd89-d89d-43bd-abb2-904413e5001d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:41.768095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84dd89-d89d-43bd-abb2-904413e5001d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:41.768597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84dd89-d89d-43bd-abb2-904413e5001d None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:41.769744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84dd89-d89d-43bd-abb2-904413e5001d None admin] RBAC: Adding query filter params (name=reader) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:41.770548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84dd89-d89d-43bd-abb2-904413e5001d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:41.772569 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84dd89-d89d-43bd-abb2-904413e5001d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:41.787228 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 171/327] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:41.788776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84dd89-d89d-43bd-abb2-904413e5001d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:41.812123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:41.822739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 157/328] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:41.850508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae93306-9aec-4d89-a445-715f0ef270e0 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:41.928421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84dd89-d89d-43bd-abb2-904413e5001d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:41.972178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f554f-dbb9-41bc-ac5d-59b39956b677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:41.973146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f554f-dbb9-41bc-ac5d-59b39956b677 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:41.973492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f554f-dbb9-41bc-ac5d-59b39956b677 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:41.973774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f554f-dbb9-41bc-ac5d-59b39956b677 None admin] PATH_INFO: `/v3/users/259572ae4cc141df9e8bdc4ebc9647eb` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:41.975115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f554f-dbb9-41bc-ac5d-59b39956b677 None admin] RBAC: Authorizing `identity:get_user(user_id=259572ae4cc141df9e8bdc4ebc9647eb)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:41.976735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f554f-dbb9-41bc-ac5d-59b39956b677 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:41.983806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 172/329] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:41 2022] GET /identity/v3/users/259572ae4cc141df9e8bdc4ebc9647eb => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:41.985705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f554f-dbb9-41bc-ac5d-59b39956b677 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:41.998624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:42.015248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae93306-9aec-4d89-a445-715f0ef270e0 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:42.074407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d0b38-cf2f-4852-933c-dbc106a9a07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:42.075376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d0b38-cf2f-4852-933c-dbc106a9a07a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:42.076183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d0b38-cf2f-4852-933c-dbc106a9a07a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:42.076183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d0b38-cf2f-4852-933c-dbc106a9a07a None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:42.090372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d0b38-cf2f-4852-933c-dbc106a9a07a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-222d0b38-cf2f-4852-933c-dbc106a9a07a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:42.100217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:42.101142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:42.101949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:42.102786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 158/330] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:34:42.108876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d0b38-cf2f-4852-933c-dbc106a9a07a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:42.113769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:42.144052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f554f-dbb9-41bc-ac5d-59b39956b677 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:42.183694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d6293-3bb3-4d22-8db3-06eaaed437fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:42.185020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d6293-3bb3-4d22-8db3-06eaaed437fa None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:42.185528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d6293-3bb3-4d22-8db3-06eaaed437fa None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:42.186031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d6293-3bb3-4d22-8db3-06eaaed437fa None admin] PATH_INFO: `/v3/system/users/259572ae4cc141df9e8bdc4ebc9647eb/roles/74ea4fc9cad8496a831bb325feb84d4c` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:42.188144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d6293-3bb3-4d22-8db3-06eaaed437fa None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=259572ae4cc141df9e8bdc4ebc9647eb, role_id=74ea4fc9cad8496a831bb325feb84d4c)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:42.189896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d6293-3bb3-4d22-8db3-06eaaed437fa None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:42.208977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 173/331] 10.222.0.70 () {58 vars in 1334 bytes} [Tue Jul 19 20:34:41 2022] PUT /identity/v3/system/users/259572ae4cc141df9e8bdc4ebc9647eb/roles/74ea4fc9cad8496a831bb325feb84d4c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:42.246732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d0b38-cf2f-4852-933c-dbc106a9a07a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:42.293387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7679df-3999-4815-a3db-27c27121baec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:42.294953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7679df-3999-4815-a3db-27c27121baec None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:42.295440 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7679df-3999-4815-a3db-27c27121baec None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:42.295864 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7679df-3999-4815-a3db-27c27121baec None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:42.296797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7679df-3999-4815-a3db-27c27121baec None admin] RBAC: Adding query filter params (name=member) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:42.297277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7679df-3999-4815-a3db-27c27121baec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:42.298861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7679df-3999-4815-a3db-27c27121baec None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:42.309820 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 159/332] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:42.318530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d6293-3bb3-4d22-8db3-06eaaed437fa None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:42.439888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d6293-3bb3-4d22-8db3-06eaaed437fa None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:42.478637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5988d-294b-4da7-b3e7-dcb6d2a9ec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:42.479634 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5988d-294b-4da7-b3e7-dcb6d2a9ec4c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:42.479926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5988d-294b-4da7-b3e7-dcb6d2a9ec4c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:42.480206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5988d-294b-4da7-b3e7-dcb6d2a9ec4c None admin] PATH_INFO: `/v3/users/b571ffab89b3479687aa4fed351fa667` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:42.481543 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5988d-294b-4da7-b3e7-dcb6d2a9ec4c None admin] RBAC: Authorizing `identity:get_user(user_id=b571ffab89b3479687aa4fed351fa667)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:42.483484 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5988d-294b-4da7-b3e7-dcb6d2a9ec4c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:42.485327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 174/333] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:42 2022] GET /identity/v3/users/b571ffab89b3479687aa4fed351fa667 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:42.491207 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7679df-3999-4815-a3db-27c27121baec None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:42.505122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:42.610149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7679df-3999-4815-a3db-27c27121baec None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:42.649921 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c6f8c-8917-4d45-9074-dd7206ee528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:42.650602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c6f8c-8917-4d45-9074-dd7206ee528d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:42.650888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c6f8c-8917-4d45-9074-dd7206ee528d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:42.651163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c6f8c-8917-4d45-9074-dd7206ee528d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:42.652015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c6f8c-8917-4d45-9074-dd7206ee528d None admin] RBAC: Adding query filter params (role.id=d94de411ec0a40be92c04950e6edcac7, user.id=b571ffab89b3479687aa4fed351fa667, scope.system=all) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:42.652328 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c6f8c-8917-4d45-9074-dd7206ee528d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:42.654387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c6f8c-8917-4d45-9074-dd7206ee528d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:42.672896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 160/334] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Jul 19 20:34:42 2022] GET /identity/v3/role_assignments?user.id=b571ffab89b3479687aa4fed351fa667&scope.system=all&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:44.968070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1fcca-5698-4b79-bc5a-a70135ee1277 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:44.968588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1fcca-5698-4b79-bc5a-a70135ee1277 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:44.968816 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1fcca-5698-4b79-bc5a-a70135ee1277 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:44.970125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 175/335] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:44.980915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797ff5c-fb5f-4292-83aa-a726440727c3 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:44.982006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797ff5c-fb5f-4292-83aa-a726440727c3 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:44.982006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797ff5c-fb5f-4292-83aa-a726440727c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.011204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-2797ff5c-fb5f-4292-83aa-a726440727c3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:45.013399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2797ff5c-fb5f-4292-83aa-a726440727c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:45.083560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7a372-e79f-463b-9172-506d6c60a024 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:45.085125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7a372-e79f-463b-9172-506d6c60a024 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:45.085125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7a372-e79f-463b-9172-506d6c60a024 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.085840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 176/336] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:45.092621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5d52-225c-45da-bcaf-19f72db6c49d None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:45.092621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5d52-225c-45da-bcaf-19f72db6c49d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:45.093001 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5d52-225c-45da-bcaf-19f72db6c49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.115873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:45.123580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a12f5d52-225c-45da-bcaf-19f72db6c49d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:45.124809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a12f5d52-225c-45da-bcaf-19f72db6c49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:45.128739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 161/337] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:45.136709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:45.231232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 177/338] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:45.232963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797ff5c-fb5f-4292-83aa-a726440727c3 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:45.247430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2eb9d-7451-4032-8de8-4a2547c60459 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:45.247731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2eb9d-7451-4032-8de8-4a2547c60459 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:45.247991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2eb9d-7451-4032-8de8-4a2547c60459 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.249690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 178/339] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:34:45.254958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024f41d-9797-40b7-8d1f-b9456fcdfe60 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:45.255248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024f41d-9797-40b7-8d1f-b9456fcdfe60 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:45.255527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024f41d-9797-40b7-8d1f-b9456fcdfe60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.283436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-7024f41d-9797-40b7-8d1f-b9456fcdfe60 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:45.284679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-7024f41d-9797-40b7-8d1f-b9456fcdfe60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:45.416996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797ff5c-fb5f-4292-83aa-a726440727c3 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:45.421576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 179/340] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:45.423348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024f41d-9797-40b7-8d1f-b9456fcdfe60 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:45.466606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa88619-5af7-49e8-8932-aa395dd1e7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:45.467424 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa88619-5af7-49e8-8932-aa395dd1e7d4 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:45.467706 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa88619-5af7-49e8-8932-aa395dd1e7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:45.467986 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa88619-5af7-49e8-8932-aa395dd1e7d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.484553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa88619-5af7-49e8-8932-aa395dd1e7d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-dfa88619-5af7-49e8-8932-aa395dd1e7d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:45.493014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:45.494021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:45.494799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:45.497575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 162/341] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:45.499252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bcb58-f3d6-408c-994c-d39c9a29dada None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:45.499749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bcb58-f3d6-408c-994c-d39c9a29dada None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:45.500219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bcb58-f3d6-408c-994c-d39c9a29dada None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.501836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 163/342] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:45.503127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bcb58-f3d6-408c-994c-d39c9a29dada None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:45.537020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:45.580271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024f41d-9797-40b7-8d1f-b9456fcdfe60 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:45.590312 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:45.636452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b7d14-b4f9-448a-aaaa-8df68db67976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:45.638342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b7d14-b4f9-448a-aaaa-8df68db67976 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:45.638851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b7d14-b4f9-448a-aaaa-8df68db67976 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:45.639332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b7d14-b4f9-448a-aaaa-8df68db67976 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.659380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bcb58-f3d6-408c-994c-d39c9a29dada None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:45.661993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b7d14-b4f9-448a-aaaa-8df68db67976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-273b7d14-b4f9-448a-aaaa-8df68db67976 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:45.671070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:45.672050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:34:45.672867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:45.676356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 180/343] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:45.681779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b7d14-b4f9-448a-aaaa-8df68db67976 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:45.703385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082277e3-562f-4fb4-afe2-cfd72e7356cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:45.704516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082277e3-562f-4fb4-afe2-cfd72e7356cc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:45.704813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082277e3-562f-4fb4-afe2-cfd72e7356cc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:45.705090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082277e3-562f-4fb4-afe2-cfd72e7356cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.721350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082277e3-562f-4fb4-afe2-cfd72e7356cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-082277e3-562f-4fb4-afe2-cfd72e7356cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:45.734425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:45.735472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:45.736304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:45.736934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 164/344] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:45.741629 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ce9b2-5709-4db7-a93f-c8dac26f7bb9 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:45.741629 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ce9b2-5709-4db7-a93f-c8dac26f7bb9 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:45.741629 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ce9b2-5709-4db7-a93f-c8dac26f7bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.761038 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-393ce9b2-5709-4db7-a93f-c8dac26f7bb9 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:45.762288 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-393ce9b2-5709-4db7-a93f-c8dac26f7bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:45.834668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b7d14-b4f9-448a-aaaa-8df68db67976 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:45.864463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 165/345] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:45.865501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ce9b2-5709-4db7-a93f-c8dac26f7bb9 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:45.874848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:45.882108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de7cc7-57f9-4307-8112-5d89c0aed71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:45.882910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de7cc7-57f9-4307-8112-5d89c0aed71f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:45.883272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de7cc7-57f9-4307-8112-5d89c0aed71f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:45.883623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de7cc7-57f9-4307-8112-5d89c0aed71f None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.884469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de7cc7-57f9-4307-8112-5d89c0aed71f None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:45.884852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de7cc7-57f9-4307-8112-5d89c0aed71f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:45.886453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de7cc7-57f9-4307-8112-5d89c0aed71f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:45.898297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 181/346] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:45.899846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd10897-6361-443f-bfd2-1c668784b8f6 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:45.900217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd10897-6361-443f-bfd2-1c668784b8f6 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:45.900643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd10897-6361-443f-bfd2-1c668784b8f6 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:45.907176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 182/347] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:45.907176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd10897-6361-443f-bfd2-1c668784b8f6 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:45.943638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:46.007637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ce9b2-5709-4db7-a93f-c8dac26f7bb9 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:46.050972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14371d52-a34e-4df8-97d5-8e416ff4c94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:46.052147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14371d52-a34e-4df8-97d5-8e416ff4c94b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:46.052703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14371d52-a34e-4df8-97d5-8e416ff4c94b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:46.052988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14371d52-a34e-4df8-97d5-8e416ff4c94b None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:46.053728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd10897-6361-443f-bfd2-1c668784b8f6 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:46.054178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14371d52-a34e-4df8-97d5-8e416ff4c94b None admin] RBAC: Adding query filter params (name=reader) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:46.054178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14371d52-a34e-4df8-97d5-8e416ff4c94b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:46.055669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14371d52-a34e-4df8-97d5-8e416ff4c94b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:46.072192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 166/348] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:46.073806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14371d52-a34e-4df8-97d5-8e416ff4c94b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:46.105795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c242407-bb00-4ad1-a5d0-9260ab9fb943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:46.106552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c242407-bb00-4ad1-a5d0-9260ab9fb943 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:46.106905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c242407-bb00-4ad1-a5d0-9260ab9fb943 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:46.107187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c242407-bb00-4ad1-a5d0-9260ab9fb943 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:46.107999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c242407-bb00-4ad1-a5d0-9260ab9fb943 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:46.108314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c242407-bb00-4ad1-a5d0-9260ab9fb943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:46.112126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c242407-bb00-4ad1-a5d0-9260ab9fb943 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:46.123667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 183/349] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:46.124700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c242407-bb00-4ad1-a5d0-9260ab9fb943 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:46.216972 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:46.228545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14371d52-a34e-4df8-97d5-8e416ff4c94b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:46.243935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:46.288007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7847e86-c9e1-4abc-b688-237f2d04517a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:46.288940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7847e86-c9e1-4abc-b688-237f2d04517a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:46.289431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7847e86-c9e1-4abc-b688-237f2d04517a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:46.289929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7847e86-c9e1-4abc-b688-237f2d04517a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:46.300410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c242407-bb00-4ad1-a5d0-9260ab9fb943 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:46.345310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14affcd-aa37-496a-b241-9ab0aa413bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:46.346422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14affcd-aa37-496a-b241-9ab0aa413bf8 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:46.346782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14affcd-aa37-496a-b241-9ab0aa413bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:46.347154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14affcd-aa37-496a-b241-9ab0aa413bf8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:46.363878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14affcd-aa37-496a-b241-9ab0aa413bf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-e14affcd-aa37-496a-b241-9ab0aa413bf8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:46.374195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:46.375171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:46.375936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:46.378626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 184/350] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:46.379866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36874c43-1b2e-4c4c-8191-d7408aeb6f75 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:46.380223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36874c43-1b2e-4c4c-8191-d7408aeb6f75 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:46.380578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36874c43-1b2e-4c4c-8191-d7408aeb6f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:46.393431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7847e86-c9e1-4abc-b688-237f2d04517a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:46.395208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7847e86-c9e1-4abc-b688-237f2d04517a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:46.404880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-36874c43-1b2e-4c4c-8191-d7408aeb6f75 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:46.406210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-36874c43-1b2e-4c4c-8191-d7408aeb6f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:46.496570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 185/351] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:46.498827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36874c43-1b2e-4c4c-8191-d7408aeb6f75 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:46.523809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 167/352] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:46.524889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7847e86-c9e1-4abc-b688-237f2d04517a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:46.538572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:46.560109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:46.631737 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36874c43-1b2e-4c4c-8191-d7408aeb6f75 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:46.671616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7847e86-c9e1-4abc-b688-237f2d04517a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:46.677312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9350a-1034-498b-8a0d-703318cf59e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:46.678703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9350a-1034-498b-8a0d-703318cf59e8 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:46.679130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9350a-1034-498b-8a0d-703318cf59e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:46.679551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9350a-1034-498b-8a0d-703318cf59e8 None admin] PATH_INFO: `/v3/users/259572ae4cc141df9e8bdc4ebc9647eb` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:46.681487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9350a-1034-498b-8a0d-703318cf59e8 None admin] RBAC: Authorizing `identity:get_user(user_id=259572ae4cc141df9e8bdc4ebc9647eb)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:46.683993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9350a-1034-498b-8a0d-703318cf59e8 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:46.691759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 186/353] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:34:46 2022] GET /identity/v3/users/259572ae4cc141df9e8bdc4ebc9647eb => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:46.691759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9350a-1034-498b-8a0d-703318cf59e8 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:46.716436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe32deb-287c-4de3-8780-31090f26431a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:46.717443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe32deb-287c-4de3-8780-31090f26431a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:46.718374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe32deb-287c-4de3-8780-31090f26431a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:46.718837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe32deb-287c-4de3-8780-31090f26431a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:46.801109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe32deb-287c-4de3-8780-31090f26431a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:46.802958 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe32deb-287c-4de3-8780-31090f26431a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:46.815608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:46.818236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9350a-1034-498b-8a0d-703318cf59e8 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:46.834718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:46.860742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81382873-4fe4-4dcb-89ae-47b60727bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:46.861468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81382873-4fe4-4dcb-89ae-47b60727bdd3 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:46.861757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81382873-4fe4-4dcb-89ae-47b60727bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:46.862030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81382873-4fe4-4dcb-89ae-47b60727bdd3 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:46.862892 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81382873-4fe4-4dcb-89ae-47b60727bdd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:46.863148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81382873-4fe4-4dcb-89ae-47b60727bdd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:46.864742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81382873-4fe4-4dcb-89ae-47b60727bdd3 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:46.880229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 187/354] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:46.880229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb126d4-f112-40cf-a380-89af945ea9fe None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:46.880229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb126d4-f112-40cf-a380-89af945ea9fe None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:46.880229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb126d4-f112-40cf-a380-89af945ea9fe None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:46.880229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 188/355] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:46.880229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb126d4-f112-40cf-a380-89af945ea9fe None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:46.932574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 168/356] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:46.935895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe32deb-287c-4de3-8780-31090f26431a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:47.005554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb126d4-f112-40cf-a380-89af945ea9fe None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:47.059103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26f0b9-c484-46a7-87ae-bb86a40fc593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:47.060726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26f0b9-c484-46a7-87ae-bb86a40fc593 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:47.061173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26f0b9-c484-46a7-87ae-bb86a40fc593 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:47.061503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26f0b9-c484-46a7-87ae-bb86a40fc593 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:47.074559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26f0b9-c484-46a7-87ae-bb86a40fc593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-cf26f0b9-c484-46a7-87ae-bb86a40fc593 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:47.083979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:47.084919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:47.085884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:47.086701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 189/357] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:47.087549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26f0b9-c484-46a7-87ae-bb86a40fc593 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:47.092192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe32deb-287c-4de3-8780-31090f26431a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:47.104553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:47.135503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461146a1-30eb-41a9-a945-687d8571dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:47.136628 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461146a1-30eb-41a9-a945-687d8571dd45 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:47.136923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461146a1-30eb-41a9-a945-687d8571dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:47.137193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461146a1-30eb-41a9-a945-687d8571dd45 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:47.139030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461146a1-30eb-41a9-a945-687d8571dd45 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:47.140497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461146a1-30eb-41a9-a945-687d8571dd45 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:47.142504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 169/358] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:46 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:47.143286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461146a1-30eb-41a9-a945-687d8571dd45 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:47.143422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:47.241511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26f0b9-c484-46a7-87ae-bb86a40fc593 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:47.286405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719b57a-219b-4ad4-96e6-04207e1a2181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:47.287386 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719b57a-219b-4ad4-96e6-04207e1a2181 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:47.287677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719b57a-219b-4ad4-96e6-04207e1a2181 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:47.287958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719b57a-219b-4ad4-96e6-04207e1a2181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:47.288862 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719b57a-219b-4ad4-96e6-04207e1a2181 None admin] RBAC: Adding query filter params (role.id=74ea4fc9cad8496a831bb325feb84d4c, user.id=259572ae4cc141df9e8bdc4ebc9647eb, scope.system=all) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:47.289192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719b57a-219b-4ad4-96e6-04207e1a2181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:47.290771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719b57a-219b-4ad4-96e6-04207e1a2181 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:47.294059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461146a1-30eb-41a9-a945-687d8571dd45 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:47.306942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 190/359] 10.222.0.70 () {56 vars in 1308 bytes} [Tue Jul 19 20:34:47 2022] GET /identity/v3/role_assignments?user.id=259572ae4cc141df9e8bdc4ebc9647eb&scope.system=all&role.id=74ea4fc9cad8496a831bb325feb84d4c => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:47.308480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c706567-d5fc-4fea-b67e-dbb7b60f1c2b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:47.308954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c706567-d5fc-4fea-b67e-dbb7b60f1c2b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:47.309489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c706567-d5fc-4fea-b67e-dbb7b60f1c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:47.328965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-8c706567-d5fc-4fea-b67e-dbb7b60f1c2b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:47.331326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8c706567-d5fc-4fea-b67e-dbb7b60f1c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:47.346450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755c8fb-3c26-4530-acf8-76a9b650ba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:47.347547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755c8fb-3c26-4530-acf8-76a9b650ba94 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:47.347838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755c8fb-3c26-4530-acf8-76a9b650ba94 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:47.348123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755c8fb-3c26-4530-acf8-76a9b650ba94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:47.420451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:47.432631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 191/360] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:47.434113 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c706567-d5fc-4fea-b67e-dbb7b60f1c2b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:47.452998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755c8fb-3c26-4530-acf8-76a9b650ba94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:47.454602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755c8fb-3c26-4530-acf8-76a9b650ba94 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:47.470049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:47.570861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c706567-d5fc-4fea-b67e-dbb7b60f1c2b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:47.604471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 170/361] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:47.605846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755c8fb-3c26-4530-acf8-76a9b650ba94 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:47.612614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e3088-5a75-4f58-b629-c5b4a4772e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:47.613637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e3088-5a75-4f58-b629-c5b4a4772e63 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:47.614050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e3088-5a75-4f58-b629-c5b4a4772e63 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:47.614477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e3088-5a75-4f58-b629-c5b4a4772e63 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:47.616364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e3088-5a75-4f58-b629-c5b4a4772e63 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:47.617462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e3088-5a75-4f58-b629-c5b4a4772e63 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:47.624731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 192/362] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:47 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:47.625505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e3088-5a75-4f58-b629-c5b4a4772e63 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:47.718685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:47.738665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:47.749865 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e3088-5a75-4f58-b629-c5b4a4772e63 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:47.751447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755c8fb-3c26-4530-acf8-76a9b650ba94 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:47.794430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a7097-e669-45df-a05c-9c1f7b4f8e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:47.795289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a7097-e669-45df-a05c-9c1f7b4f8e5c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:47.795565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a7097-e669-45df-a05c-9c1f7b4f8e5c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:47.797872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a7097-e669-45df-a05c-9c1f7b4f8e5c None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:47.798305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffda4a7-29c5-42de-96f3-0ed853e2402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:47.798983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffda4a7-29c5-42de-96f3-0ed853e2402a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:47.799268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffda4a7-29c5-42de-96f3-0ed853e2402a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:47.799545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffda4a7-29c5-42de-96f3-0ed853e2402a None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:47.800314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffda4a7-29c5-42de-96f3-0ed853e2402a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:47.800626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffda4a7-29c5-42de-96f3-0ed853e2402a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:47.802049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffda4a7-29c5-42de-96f3-0ed853e2402a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:47.802841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a7097-e669-45df-a05c-9c1f7b4f8e5c None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:47.804808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a7097-e669-45df-a05c-9c1f7b4f8e5c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:47.806776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 193/363] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:47 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:47.807478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a7097-e669-45df-a05c-9c1f7b4f8e5c None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:47.814677 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 171/364] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:47.815578 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffda4a7-29c5-42de-96f3-0ed853e2402a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:47.910289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a7097-e669-45df-a05c-9c1f7b4f8e5c None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:47.919844 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffda4a7-29c5-42de-96f3-0ed853e2402a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:47.945313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c1b6-42b6-4264-a1b9-0f54f3020bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:47.946148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c1b6-42b6-4264-a1b9-0f54f3020bd2 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:47.946450 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c1b6-42b6-4264-a1b9-0f54f3020bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:47.946725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c1b6-42b6-4264-a1b9-0f54f3020bd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:47.954980 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0115a-74be-42c4-b461-b96abd1f420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:47.955833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0115a-74be-42c4-b461-b96abd1f420c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:47.956115 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0115a-74be-42c4-b461-b96abd1f420c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:47.956400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0115a-74be-42c4-b461-b96abd1f420c None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:47.957168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351c1b6-42b6-4264-a1b9-0f54f3020bd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:47.957598 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0115a-74be-42c4-b461-b96abd1f420c None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:47.959049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0115a-74be-42c4-b461-b96abd1f420c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:47.960740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 172/365] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:47 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:47.961512 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0115a-74be-42c4-b461-b96abd1f420c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-1351c1b6-42b6-4264-a1b9-0f54f3020bd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:47.965187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:47.966224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:47.967003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:47.969513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 194/366] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:34:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:47.970523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c1b6-42b6-4264-a1b9-0f54f3020bd2 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:47.975399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:47.989887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:48.067310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0115a-74be-42c4-b461-b96abd1f420c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:48.078643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c1b6-42b6-4264-a1b9-0f54f3020bd2 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:48.101679 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa38e20-ad1b-4e1a-8966-c07dad1f47c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:48.102544 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa38e20-ad1b-4e1a-8966-c07dad1f47c1 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:48.102825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa38e20-ad1b-4e1a-8966-c07dad1f47c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:48.103097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa38e20-ad1b-4e1a-8966-c07dad1f47c1 None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:48.104262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa38e20-ad1b-4e1a-8966-c07dad1f47c1 None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:48.105739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa38e20-ad1b-4e1a-8966-c07dad1f47c1 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:48.107866 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 173/367] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:47 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:48.108617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa38e20-ad1b-4e1a-8966-c07dad1f47c1 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:48.115827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5272c79-9a48-427c-9c0f-1c2b6bd2acc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:48.116494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5272c79-9a48-427c-9c0f-1c2b6bd2acc0 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:48.116783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5272c79-9a48-427c-9c0f-1c2b6bd2acc0 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:48.117065 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5272c79-9a48-427c-9c0f-1c2b6bd2acc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:48.117941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5272c79-9a48-427c-9c0f-1c2b6bd2acc0 None admin] RBAC: Adding query filter params (group.id=7cde2658873347688bffa512ef1b112e, role.id=d94de411ec0a40be92c04950e6edcac7, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:48.118261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5272c79-9a48-427c-9c0f-1c2b6bd2acc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:48.119710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5272c79-9a48-427c-9c0f-1c2b6bd2acc0 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:48.126443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 195/368] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jul 19 20:34:47 2022] GET /identity/v3/role_assignments?group.id=7cde2658873347688bffa512ef1b112e&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:48.127252 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5272c79-9a48-427c-9c0f-1c2b6bd2acc0 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:48.206519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:48.236973 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:48.238721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa38e20-ad1b-4e1a-8966-c07dad1f47c1 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:48.264467 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5272c79-9a48-427c-9c0f-1c2b6bd2acc0 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:48.279200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2db917-f8c6-4d44-af33-f30f0f30387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:48.279870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2db917-f8c6-4d44-af33-f30f0f30387c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:48.280146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2db917-f8c6-4d44-af33-f30f0f30387c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:48.280424 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2db917-f8c6-4d44-af33-f30f0f30387c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:48.307388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd10395-26e9-4a22-9627-ab7d3925ca5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:48.308257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd10395-26e9-4a22-9627-ab7d3925ca5d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:48.308544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd10395-26e9-4a22-9627-ab7d3925ca5d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:48.308826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd10395-26e9-4a22-9627-ab7d3925ca5d None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:48.310063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd10395-26e9-4a22-9627-ab7d3925ca5d None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:48.311518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd10395-26e9-4a22-9627-ab7d3925ca5d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:48.313173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 196/369] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:48 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:48.314006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd10395-26e9-4a22-9627-ab7d3925ca5d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:48.370573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2db917-f8c6-4d44-af33-f30f0f30387c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:48.372181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2db917-f8c6-4d44-af33-f30f0f30387c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:48.430528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd10395-26e9-4a22-9627-ab7d3925ca5d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:48.462967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:48.471906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49ea1-6f63-4f0e-af01-84b1ec4603cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:48.472600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49ea1-6f63-4f0e-af01-84b1ec4603cb None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:48.472891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49ea1-6f63-4f0e-af01-84b1ec4603cb None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:48.473173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb49ea1-6f63-4f0e-af01-84b1ec4603cb None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:48.473983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb49ea1-6f63-4f0e-af01-84b1ec4603cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:48.474295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb49ea1-6f63-4f0e-af01-84b1ec4603cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:48.475705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb49ea1-6f63-4f0e-af01-84b1ec4603cb None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:48.485046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 197/370] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:48.485933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49ea1-6f63-4f0e-af01-84b1ec4603cb None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:48.496097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 174/371] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:48.497295 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2db917-f8c6-4d44-af33-f30f0f30387c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:48.506524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:48.612073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb49ea1-6f63-4f0e-af01-84b1ec4603cb None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:48.630393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2db917-f8c6-4d44-af33-f30f0f30387c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:48.650589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10732c6d-8eae-4722-8702-bb5bef9af51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:48.651443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10732c6d-8eae-4722-8702-bb5bef9af51e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:48.651871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10732c6d-8eae-4722-8702-bb5bef9af51e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:48.652350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10732c6d-8eae-4722-8702-bb5bef9af51e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:48.654201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10732c6d-8eae-4722-8702-bb5bef9af51e None admin] RBAC: Adding query filter params (group.id=7cde2658873347688bffa512ef1b112e, role.id=0044294fcceb4b6ba0cb27567f2761d4, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:48.654683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10732c6d-8eae-4722-8702-bb5bef9af51e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:48.656394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10732c6d-8eae-4722-8702-bb5bef9af51e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:48.673395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 198/372] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jul 19 20:34:48 2022] GET /identity/v3/role_assignments?group.id=7cde2658873347688bffa512ef1b112e&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=0044294fcceb4b6ba0cb27567f2761d4 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:48.673395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10732c6d-8eae-4722-8702-bb5bef9af51e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:48.688409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8173a7-cf6b-409a-af1c-fb72fdb6bc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:48.689112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8173a7-cf6b-409a-af1c-fb72fdb6bc7a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:48.689499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8173a7-cf6b-409a-af1c-fb72fdb6bc7a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:48.689786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8173a7-cf6b-409a-af1c-fb72fdb6bc7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:48.690640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8173a7-cf6b-409a-af1c-fb72fdb6bc7a None admin] RBAC: Adding query filter params (group.id=7cde2658873347688bffa512ef1b112e, role.id=d94de411ec0a40be92c04950e6edcac7, scope.project.id=0d8860ce338747a5ba3d5903b48cc986) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:48.690953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8173a7-cf6b-409a-af1c-fb72fdb6bc7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:48.692428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8173a7-cf6b-409a-af1c-fb72fdb6bc7a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:48.699434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 175/373] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jul 19 20:34:48 2022] GET /identity/v3/role_assignments?group.id=7cde2658873347688bffa512ef1b112e&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:48.710545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8173a7-cf6b-409a-af1c-fb72fdb6bc7a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:48.825878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:48.861462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:48.905387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8173a7-cf6b-409a-af1c-fb72fdb6bc7a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:48.910590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10732c6d-8eae-4722-8702-bb5bef9af51e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:48.945891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2513d-407e-4278-b172-f1ea32b797e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:48.946796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2513d-407e-4278-b172-f1ea32b797e9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:48.947213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2513d-407e-4278-b172-f1ea32b797e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:48.947493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2513d-407e-4278-b172-f1ea32b797e9 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:48.948995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2513d-407e-4278-b172-f1ea32b797e9 None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:48.951070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2513d-407e-4278-b172-f1ea32b797e9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:48.953111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 176/374] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:48 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:48.966628 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2513d-407e-4278-b172-f1ea32b797e9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:49.006593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f974e-566e-4705-b583-3823d1b4453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:49.007747 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f974e-566e-4705-b583-3823d1b4453b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:49.008268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f974e-566e-4705-b583-3823d1b4453b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:49.008959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f974e-566e-4705-b583-3823d1b4453b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:49.087220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2513d-407e-4278-b172-f1ea32b797e9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:49.098149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:49.128387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ac0e7-8a89-46c8-8af1-921f50f89eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:49.129631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ac0e7-8a89-46c8-8af1-921f50f89eb2 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:49.130097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ac0e7-8a89-46c8-8af1-921f50f89eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:49.130460 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ac0e7-8a89-46c8-8af1-921f50f89eb2 None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:49.131939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ac0e7-8a89-46c8-8af1-921f50f89eb2 None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:49.133760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ac0e7-8a89-46c8-8af1-921f50f89eb2 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:49.135657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 177/375] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:48 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:49.143405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ac0e7-8a89-46c8-8af1-921f50f89eb2 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:49.202296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f974e-566e-4705-b583-3823d1b4453b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:49.203769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f974e-566e-4705-b583-3823d1b4453b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:49.264517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ac0e7-8a89-46c8-8af1-921f50f89eb2 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:49.318997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc3403-7102-445d-8212-0d21bc120e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:49.319839 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc3403-7102-445d-8212-0d21bc120e42 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:49.321654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc3403-7102-445d-8212-0d21bc120e42 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:49.322281 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc3403-7102-445d-8212-0d21bc120e42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:49.323429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc3403-7102-445d-8212-0d21bc120e42 None admin] RBAC: Adding query filter params (group.id=7cde2658873347688bffa512ef1b112e, role.id=0044294fcceb4b6ba0cb27567f2761d4, scope.project.id=0d8860ce338747a5ba3d5903b48cc986) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:49.324415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc3403-7102-445d-8212-0d21bc120e42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:49.326874 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc3403-7102-445d-8212-0d21bc120e42 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:49.351308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 178/376] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jul 19 20:34:49 2022] GET /identity/v3/role_assignments?group.id=7cde2658873347688bffa512ef1b112e&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=0044294fcceb4b6ba0cb27567f2761d4 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:49.392336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:49.445827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 199/377] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 781 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:49.452266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc3403-7102-445d-8212-0d21bc120e42 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:49.522636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:49.591646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc3403-7102-445d-8212-0d21bc120e42 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:49.628599 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb3b71-2af0-4910-b362-406502816739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:49.629423 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3b71-2af0-4910-b362-406502816739 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:49.629713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3b71-2af0-4910-b362-406502816739 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:49.629993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb3b71-2af0-4910-b362-406502816739 None admin] PATH_INFO: `/v3/projects/a9ddf9c6d32c459d8f135c30485e0f10` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:49.631241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb3b71-2af0-4910-b362-406502816739 None admin] RBAC: Authorizing `identity:get_project(project_id=a9ddf9c6d32c459d8f135c30485e0f10)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:49.632705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb3b71-2af0-4910-b362-406502816739 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:49.634806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 179/378] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:49 2022] GET /identity/v3/projects/a9ddf9c6d32c459d8f135c30485e0f10 => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:49.640826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f974e-566e-4705-b583-3823d1b4453b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:49.909046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f974e-566e-4705-b583-3823d1b4453b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:49.967664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbc867-6edc-42ca-9375-d27087bfbf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:49.969247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbc867-6edc-42ca-9375-d27087bfbf53 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:49.969784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbc867-6edc-42ca-9375-d27087bfbf53 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:49.970336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbc867-6edc-42ca-9375-d27087bfbf53 None admin] PATH_INFO: `/v3/groups/34c9049ade024859804f8298dbfc8db3` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:49.978640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbc867-6edc-42ca-9375-d27087bfbf53 None admin] RBAC: Authorizing `identity:get_group(group_id=34c9049ade024859804f8298dbfc8db3)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:49.981959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbc867-6edc-42ca-9375-d27087bfbf53 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:49.985335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 200/379] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:49 2022] GET /identity/v3/groups/34c9049ade024859804f8298dbfc8db3 => generated 229 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:49.990880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb3b71-2af0-4910-b362-406502816739 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:50.120543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb3b71-2af0-4910-b362-406502816739 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:50.130273 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:50.157258 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2579a-a3d2-44a5-9dc3-21a657282514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:50.157966 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2579a-a3d2-44a5-9dc3-21a657282514 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:50.158255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2579a-a3d2-44a5-9dc3-21a657282514 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:50.158538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2579a-a3d2-44a5-9dc3-21a657282514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:50.159500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2579a-a3d2-44a5-9dc3-21a657282514 None admin] RBAC: Adding query filter params (group.id=34c9049ade024859804f8298dbfc8db3, role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=a9ddf9c6d32c459d8f135c30485e0f10) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:50.159761 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2579a-a3d2-44a5-9dc3-21a657282514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:50.161259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2579a-a3d2-44a5-9dc3-21a657282514 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:50.167773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 180/380] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jul 19 20:34:49 2022] GET /identity/v3/role_assignments?group.id=34c9049ade024859804f8298dbfc8db3&scope.project.id=a9ddf9c6d32c459d8f135c30485e0f10&role.id=85420c68300642aabfd730fb8fef8347 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:50.920996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321af1b-fd4b-462e-9450-8164905aae1a None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:50.921588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321af1b-fd4b-462e-9450-8164905aae1a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:50.921588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321af1b-fd4b-462e-9450-8164905aae1a None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:50.923095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 201/381] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:50.928645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d509-eb3f-4665-b6b7-7da2e929b7c4 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:50.928946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d509-eb3f-4665-b6b7-7da2e929b7c4 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:50.929234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160d509-eb3f-4665-b6b7-7da2e929b7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:50.950272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c160d509-eb3f-4665-b6b7-7da2e929b7c4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:50.951489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c160d509-eb3f-4665-b6b7-7da2e929b7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:51.048149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 181/382] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:51.083466 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321af1b-fd4b-462e-9450-8164905aae1a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:51.137623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:51.226187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321af1b-fd4b-462e-9450-8164905aae1a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:51.248306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd841c-6e3b-42da-8adc-a4066c83407a None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:51.249259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd841c-6e3b-42da-8adc-a4066c83407a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:51.249787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd841c-6e3b-42da-8adc-a4066c83407a None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:51.255180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 182/383] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:34:51.256747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4374a3-813c-4138-8705-a7290482ce81 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:51.257240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4374a3-813c-4138-8705-a7290482ce81 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:51.257742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4374a3-813c-4138-8705-a7290482ce81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:51.285520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4f848-b24c-4ff0-87c6-c9a3759c985e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:51.286490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4f848-b24c-4ff0-87c6-c9a3759c985e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:51.286782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4f848-b24c-4ff0-87c6-c9a3759c985e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:51.287050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4f848-b24c-4ff0-87c6-c9a3759c985e None admin] PATH_INFO: `/v3/roles/member` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:51.288487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-4f4374a3-813c-4138-8705-a7290482ce81 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:51.289722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4f4374a3-813c-4138-8705-a7290482ce81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:51.313138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4f848-b24c-4ff0-87c6-c9a3759c985e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-93e4f848-b24c-4ff0-87c6-c9a3759c985e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:51.321071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:51.322009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:51.323052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 202/384] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:51.328768 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4f848-b24c-4ff0-87c6-c9a3759c985e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:51.444968 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 183/385] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:51.470192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4374a3-813c-4138-8705-a7290482ce81 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:51.491765 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:51.503198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4f848-b24c-4ff0-87c6-c9a3759c985e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:51.517258 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:51.571955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8b2bb-48bb-42f4-b162-6b8548096d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:51.576476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8b2bb-48bb-42f4-b162-6b8548096d2e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:51.576986 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8b2bb-48bb-42f4-b162-6b8548096d2e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:51.577494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8b2bb-48bb-42f4-b162-6b8548096d2e None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:51.578482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8b2bb-48bb-42f4-b162-6b8548096d2e None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:51.578997 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8b2bb-48bb-42f4-b162-6b8548096d2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:51.580575 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8b2bb-48bb-42f4-b162-6b8548096d2e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:51.597288 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 203/386] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:51.605416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8b2bb-48bb-42f4-b162-6b8548096d2e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:51.641830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4374a3-813c-4138-8705-a7290482ce81 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:51.705199 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5928e2-00e6-4b21-82ed-bb073424a9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:51.706057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5928e2-00e6-4b21-82ed-bb073424a9d7 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:51.706336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5928e2-00e6-4b21-82ed-bb073424a9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:51.706708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5928e2-00e6-4b21-82ed-bb073424a9d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:51.718217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5928e2-00e6-4b21-82ed-bb073424a9d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-4e5928e2-00e6-4b21-82ed-bb073424a9d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:51.726941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:51.727830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:51.728627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 184/387] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:34:51.730073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749d69e-1396-48ad-b958-f8b0c1b6da92 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:51.730073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749d69e-1396-48ad-b958-f8b0c1b6da92 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:51.730244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749d69e-1396-48ad-b958-f8b0c1b6da92 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:51.732425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 185/388] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:51.741837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749d69e-1396-48ad-b958-f8b0c1b6da92 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:51.767027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8b2bb-48bb-42f4-b162-6b8548096d2e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:51.812208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d09297-fdff-4393-87ec-102a90025134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:51.813655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d09297-fdff-4393-87ec-102a90025134 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:51.814151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d09297-fdff-4393-87ec-102a90025134 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:51.814560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d09297-fdff-4393-87ec-102a90025134 None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:51.815752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d09297-fdff-4393-87ec-102a90025134 None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:51.817193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d09297-fdff-4393-87ec-102a90025134 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:51.819435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 204/389] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:51 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:51.821812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9abad-764c-4484-864b-09cea8085855 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:51.822109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9abad-764c-4484-864b-09cea8085855 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:51.822392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9abad-764c-4484-864b-09cea8085855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:51.843411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-cce9abad-764c-4484-864b-09cea8085855 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:51.844925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-cce9abad-764c-4484-864b-09cea8085855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:51.883866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:51.899457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:51.912813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749d69e-1396-48ad-b958-f8b0c1b6da92 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:51.950144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 205/390] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:51.951189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9abad-764c-4484-864b-09cea8085855 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:51.974313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68a84e-2e27-4485-8c88-442bae43a077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:51.975013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68a84e-2e27-4485-8c88-442bae43a077 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:51.975305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68a84e-2e27-4485-8c88-442bae43a077 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:51.975588 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68a84e-2e27-4485-8c88-442bae43a077 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:51.976359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68a84e-2e27-4485-8c88-442bae43a077 None admin] RBAC: Adding query filter params (name=member) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:51.976668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68a84e-2e27-4485-8c88-442bae43a077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:51.978095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68a84e-2e27-4485-8c88-442bae43a077 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:51.991650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 186/391] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:51.994709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68a84e-2e27-4485-8c88-442bae43a077 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:52.071569 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9abad-764c-4484-864b-09cea8085855 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:52.101219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68a84e-2e27-4485-8c88-442bae43a077 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:52.112294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffe396-b7b0-4cfc-a50b-7fe0b436c79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:52.113386 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffe396-b7b0-4cfc-a50b-7fe0b436c79b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.113694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffe396-b7b0-4cfc-a50b-7fe0b436c79b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.113974 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffe396-b7b0-4cfc-a50b-7fe0b436c79b None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.115251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffe396-b7b0-4cfc-a50b-7fe0b436c79b None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:52.116942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffe396-b7b0-4cfc-a50b-7fe0b436c79b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:52.119183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 206/392] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:51 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:52.120202 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffe396-b7b0-4cfc-a50b-7fe0b436c79b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:52.137880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400c116-b41d-415b-9032-f394f0e8cc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:52.138731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400c116-b41d-415b-9032-f394f0e8cc07 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.139012 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400c116-b41d-415b-9032-f394f0e8cc07 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.139284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400c116-b41d-415b-9032-f394f0e8cc07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.150509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400c116-b41d-415b-9032-f394f0e8cc07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:52.151533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-4400c116-b41d-415b-9032-f394f0e8cc07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:52.161143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:52.163435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:52.165421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:52.173491 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 187/393] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:34:52.173491 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400c116-b41d-415b-9032-f394f0e8cc07 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:52.191188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:52.248983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffe396-b7b0-4cfc-a50b-7fe0b436c79b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:52.289001 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a21d1-b786-4e1c-a58f-1a51cb2ae7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:52.289910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a21d1-b786-4e1c-a58f-1a51cb2ae7fb None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.290194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a21d1-b786-4e1c-a58f-1a51cb2ae7fb None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.290471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a21d1-b786-4e1c-a58f-1a51cb2ae7fb None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.291469 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400c116-b41d-415b-9032-f394f0e8cc07 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:52.291940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a21d1-b786-4e1c-a58f-1a51cb2ae7fb None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:52.293261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a21d1-b786-4e1c-a58f-1a51cb2ae7fb None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:52.300281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 207/394] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:52 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:52.302015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a21d1-b786-4e1c-a58f-1a51cb2ae7fb None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:52.334256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e252826-ab0e-49b1-9a1d-1af70b2f4b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:52.335587 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e252826-ab0e-49b1-9a1d-1af70b2f4b9f None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.336032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e252826-ab0e-49b1-9a1d-1af70b2f4b9f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.336462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e252826-ab0e-49b1-9a1d-1af70b2f4b9f None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/groups/7cde2658873347688bffa512ef1b112e/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.339365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e252826-ab0e-49b1-9a1d-1af70b2f4b9f None admin] RBAC: Authorizing `identity:create_grant(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc, group_id=7cde2658873347688bffa512ef1b112e, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:52.341708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e252826-ab0e-49b1-9a1d-1af70b2f4b9f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:52.358784 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 188/395] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Jul 19 20:34:52 2022] PUT /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/groups/7cde2658873347688bffa512ef1b112e/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:52.360107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e252826-ab0e-49b1-9a1d-1af70b2f4b9f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:52.436244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:52.444236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a21d1-b786-4e1c-a58f-1a51cb2ae7fb None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:52.460260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:52.493115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace7cda-2e39-48f2-835b-28fd4e230185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:52.494160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace7cda-2e39-48f2-835b-28fd4e230185 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.494318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e252826-ab0e-49b1-9a1d-1af70b2f4b9f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:52.494808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace7cda-2e39-48f2-835b-28fd4e230185 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.495402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace7cda-2e39-48f2-835b-28fd4e230185 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.496745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace7cda-2e39-48f2-835b-28fd4e230185 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:52.497955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace7cda-2e39-48f2-835b-28fd4e230185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:52.499915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace7cda-2e39-48f2-835b-28fd4e230185 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:52.517022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 208/396] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:52.519496 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cdb04-fe7d-4a22-9596-f5e0bb8f9c81 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.519985 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cdb04-fe7d-4a22-9596-f5e0bb8f9c81 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cdb04-fe7d-4a22-9596-f5e0bb8f9c81 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.522501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 209/397] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:52.523563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cdb04-fe7d-4a22-9596-f5e0bb8f9c81 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:52.543123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f866c5-c454-4c4a-b535-666d3a800fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:52.544291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f866c5-c454-4c4a-b535-666d3a800fd8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.544576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f866c5-c454-4c4a-b535-666d3a800fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.544853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f866c5-c454-4c4a-b535-666d3a800fd8 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.546285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f866c5-c454-4c4a-b535-666d3a800fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:52.547771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f866c5-c454-4c4a-b535-666d3a800fd8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:52.549969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 189/398] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:52 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:52.551393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a67043-b6b1-4125-a02f-206f8d1f5f2b None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.552013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a67043-b6b1-4125-a02f-206f8d1f5f2b None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.552013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a67043-b6b1-4125-a02f-206f8d1f5f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.576035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-85a67043-b6b1-4125-a02f-206f8d1f5f2b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:52.578271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-85a67043-b6b1-4125-a02f-206f8d1f5f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:52.681883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 190/399] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:52.683087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a67043-b6b1-4125-a02f-206f8d1f5f2b None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:52.696114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cdb04-fe7d-4a22-9596-f5e0bb8f9c81 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:52.744245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1f66e-5bea-47b9-bb46-6a3ac788cb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:52.745183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1f66e-5bea-47b9-bb46-6a3ac788cb7b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.745517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1f66e-5bea-47b9-bb46-6a3ac788cb7b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.745799 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1f66e-5bea-47b9-bb46-6a3ac788cb7b None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.747793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1f66e-5bea-47b9-bb46-6a3ac788cb7b None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:52.749654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1f66e-5bea-47b9-bb46-6a3ac788cb7b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:52.751696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 210/400] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:52 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:52.753911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfda73-a7d7-414e-9447-2536c65af90f None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.754412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfda73-a7d7-414e-9447-2536c65af90f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.754904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfda73-a7d7-414e-9447-2536c65af90f None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.761577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 211/401] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:52.763860 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfda73-a7d7-414e-9447-2536c65af90f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:52.792501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:52.798299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:52.837047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a67043-b6b1-4125-a02f-206f8d1f5f2b None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:52.877918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6096c-febc-4b57-b930-7ccd4b2a6e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:52.878808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6096c-febc-4b57-b930-7ccd4b2a6e1e None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.879093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6096c-febc-4b57-b930-7ccd4b2a6e1e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.879367 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6096c-febc-4b57-b930-7ccd4b2a6e1e None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986/groups/7cde2658873347688bffa512ef1b112e/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.881643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6096c-febc-4b57-b930-7ccd4b2a6e1e None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8860ce338747a5ba3d5903b48cc986, group_id=7cde2658873347688bffa512ef1b112e, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:52.883202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6096c-febc-4b57-b930-7ccd4b2a6e1e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:52.885430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfda73-a7d7-414e-9447-2536c65af90f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:52.909399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 191/402] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Jul 19 20:34:52 2022] PUT /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986/groups/7cde2658873347688bffa512ef1b112e/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:52.909399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6096c-febc-4b57-b930-7ccd4b2a6e1e None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:52.939572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa67e5a-fbf6-4d3e-93ae-c4b9eb05603d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:52.940959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa67e5a-fbf6-4d3e-93ae-c4b9eb05603d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.941296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa67e5a-fbf6-4d3e-93ae-c4b9eb05603d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.941623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa67e5a-fbf6-4d3e-93ae-c4b9eb05603d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:52.956536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa67e5a-fbf6-4d3e-93ae-c4b9eb05603d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-7aa67e5a-fbf6-4d3e-93ae-c4b9eb05603d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:52.966611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:52.967591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:52.968357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 212/403] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:52.975184 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e81f9a-4bcb-491a-82a0-ba9081be678c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:52.975496 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e81f9a-4bcb-491a-82a0-ba9081be678c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:52.975765 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e81f9a-4bcb-491a-82a0-ba9081be678c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:53.000382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-81e81f9a-4bcb-491a-82a0-ba9081be678c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:53.001759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-81e81f9a-4bcb-491a-82a0-ba9081be678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:53.022256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6096c-febc-4b57-b930-7ccd4b2a6e1e None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:53.058215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:53.067236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfea5d-f166-4b8b-90bb-b5c05cf45cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:53.068119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfea5d-f166-4b8b-90bb-b5c05cf45cc2 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:53.068917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfea5d-f166-4b8b-90bb-b5c05cf45cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:53.069667 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfea5d-f166-4b8b-90bb-b5c05cf45cc2 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:53.071491 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfea5d-f166-4b8b-90bb-b5c05cf45cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:53.073294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfea5d-f166-4b8b-90bb-b5c05cf45cc2 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:53.076295 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 192/404] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:52 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:53.077073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfea5d-f166-4b8b-90bb-b5c05cf45cc2 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:53.092305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 213/405] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:53.093390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e81f9a-4bcb-491a-82a0-ba9081be678c None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:53.112780 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:53.198240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfea5d-f166-4b8b-90bb-b5c05cf45cc2 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:53.227183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e81f9a-4bcb-491a-82a0-ba9081be678c None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:53.238721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bb802-9645-4964-94b6-186908eecc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:53.239660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bb802-9645-4964-94b6-186908eecc9b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:53.240130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bb802-9645-4964-94b6-186908eecc9b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:53.240554 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bb802-9645-4964-94b6-186908eecc9b None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:53.241503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bb802-9645-4964-94b6-186908eecc9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:53.241966 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bb802-9645-4964-94b6-186908eecc9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:53.243705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bb802-9645-4964-94b6-186908eecc9b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:53.257097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 193/406] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:53.262987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bb802-9645-4964-94b6-186908eecc9b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:53.280052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d4dae-fe65-4b23-b82f-01394b544d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:53.281810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d4dae-fe65-4b23-b82f-01394b544d8f None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:53.282246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d4dae-fe65-4b23-b82f-01394b544d8f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:53.283440 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d4dae-fe65-4b23-b82f-01394b544d8f None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/groups/7cde2658873347688bffa512ef1b112e/roles/0044294fcceb4b6ba0cb27567f2761d4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:53.289895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d4dae-fe65-4b23-b82f-01394b544d8f None admin] RBAC: Authorizing `identity:create_grant(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc, group_id=7cde2658873347688bffa512ef1b112e, role_id=0044294fcceb4b6ba0cb27567f2761d4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:53.291874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d4dae-fe65-4b23-b82f-01394b544d8f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:53.314626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 214/407] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Jul 19 20:34:53 2022] PUT /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc/groups/7cde2658873347688bffa512ef1b112e/roles/0044294fcceb4b6ba0cb27567f2761d4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:53.315794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d4dae-fe65-4b23-b82f-01394b544d8f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:53.402435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:53.453757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:53.501394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bb802-9645-4964-94b6-186908eecc9b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:53.525712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d4dae-fe65-4b23-b82f-01394b544d8f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:53.555532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85a64b-f14d-4799-8664-12ed2e256140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:53.556351 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85a64b-f14d-4799-8664-12ed2e256140 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:53.556732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85a64b-f14d-4799-8664-12ed2e256140 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:53.557017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85a64b-f14d-4799-8664-12ed2e256140 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:53.570546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85a64b-f14d-4799-8664-12ed2e256140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-ab85a64b-f14d-4799-8664-12ed2e256140 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:53.583173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:53.584111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:53.584911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 194/408] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:34:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:53.593844 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85a64b-f14d-4799-8664-12ed2e256140 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:53.599101 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21bff2-0163-46cf-b4a5-dfeafd0139c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:53.601978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21bff2-0163-46cf-b4a5-dfeafd0139c6 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:53.602292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21bff2-0163-46cf-b4a5-dfeafd0139c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:53.602573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21bff2-0163-46cf-b4a5-dfeafd0139c6 None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:53.604128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21bff2-0163-46cf-b4a5-dfeafd0139c6 None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:53.605749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21bff2-0163-46cf-b4a5-dfeafd0139c6 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:53.610427 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 215/409] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:53 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:53.623460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21bff2-0163-46cf-b4a5-dfeafd0139c6 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:53.791513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21bff2-0163-46cf-b4a5-dfeafd0139c6 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:53.836507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85a64b-f14d-4799-8664-12ed2e256140 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:53.875515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b4baa-f67f-43ef-b17e-1fb3115dcc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:53.876390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b4baa-f67f-43ef-b17e-1fb3115dcc3e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:53.876721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b4baa-f67f-43ef-b17e-1fb3115dcc3e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:53.876957 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b4baa-f67f-43ef-b17e-1fb3115dcc3e None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:53.879975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b4baa-f67f-43ef-b17e-1fb3115dcc3e None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:53.881999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b4baa-f67f-43ef-b17e-1fb3115dcc3e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:53.885725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 216/410] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:53 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:53.886145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:53.890364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b4baa-f67f-43ef-b17e-1fb3115dcc3e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:53.911370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd992d1a-e716-4c17-89ff-3ab806a4e95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:53.912048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992d1a-e716-4c17-89ff-3ab806a4e95f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:53.912340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992d1a-e716-4c17-89ff-3ab806a4e95f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:53.912604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992d1a-e716-4c17-89ff-3ab806a4e95f None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:53.913375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd992d1a-e716-4c17-89ff-3ab806a4e95f None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:53.913692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd992d1a-e716-4c17-89ff-3ab806a4e95f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:53.913859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:53.915109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd992d1a-e716-4c17-89ff-3ab806a4e95f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:53.935991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 195/411] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:53.939836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd992d1a-e716-4c17-89ff-3ab806a4e95f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:54.075305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b4baa-f67f-43ef-b17e-1fb3115dcc3e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:54.138220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd992d1a-e716-4c17-89ff-3ab806a4e95f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:54.155234 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c9b61-8719-4b93-b3f6-080c0d4be2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:54.156140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c9b61-8719-4b93-b3f6-080c0d4be2f4 None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:54.156421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c9b61-8719-4b93-b3f6-080c0d4be2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:54.156695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c9b61-8719-4b93-b3f6-080c0d4be2f4 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986/groups/7cde2658873347688bffa512ef1b112e/roles/0044294fcceb4b6ba0cb27567f2761d4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:54.162640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c9b61-8719-4b93-b3f6-080c0d4be2f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d8860ce338747a5ba3d5903b48cc986, group_id=7cde2658873347688bffa512ef1b112e, role_id=0044294fcceb4b6ba0cb27567f2761d4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:54.164293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c9b61-8719-4b93-b3f6-080c0d4be2f4 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:54.176813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3bd5b-1da8-47cb-b631-f91c7cafb5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:54.177839 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3bd5b-1da8-47cb-b631-f91c7cafb5d0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:54.178110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3bd5b-1da8-47cb-b631-f91c7cafb5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:54.178383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3bd5b-1da8-47cb-b631-f91c7cafb5d0 None admin] PATH_INFO: `/v3/groups/34c9049ade024859804f8298dbfc8db3` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:54.179608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3bd5b-1da8-47cb-b631-f91c7cafb5d0 None admin] RBAC: Authorizing `identity:get_group(group_id=34c9049ade024859804f8298dbfc8db3)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:54.181074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3bd5b-1da8-47cb-b631-f91c7cafb5d0 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:54.183201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 196/412] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:53 2022] GET /identity/v3/groups/34c9049ade024859804f8298dbfc8db3 => generated 229 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:54.192341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3bd5b-1da8-47cb-b631-f91c7cafb5d0 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:54.199215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 217/413] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Jul 19 20:34:53 2022] PUT /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986/groups/7cde2658873347688bffa512ef1b112e/roles/0044294fcceb4b6ba0cb27567f2761d4 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:54.261166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:54.317513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3bd5b-1da8-47cb-b631-f91c7cafb5d0 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:54.357191 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134ddcc-6680-4ff9-8e5a-83363e82b7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:54.358352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ddcc-6680-4ff9-8e5a-83363e82b7cb None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:54.358791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ddcc-6680-4ff9-8e5a-83363e82b7cb None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:54.359214 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ddcc-6680-4ff9-8e5a-83363e82b7cb None admin] PATH_INFO: `/v3/projects/a9ddf9c6d32c459d8f135c30485e0f10` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:54.360745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134ddcc-6680-4ff9-8e5a-83363e82b7cb None admin] RBAC: Authorizing `identity:get_project(project_id=a9ddf9c6d32c459d8f135c30485e0f10)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:54.362478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134ddcc-6680-4ff9-8e5a-83363e82b7cb None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:54.364716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 197/414] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:54 2022] GET /identity/v3/projects/a9ddf9c6d32c459d8f135c30485e0f10 => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:34:54.381432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c9b61-8719-4b93-b3f6-080c0d4be2f4 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:54.473812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:54.511212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c9b61-8719-4b93-b3f6-080c0d4be2f4 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:54.559886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7f74d-2f27-4665-8fd4-4ad8949bca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:54.561106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f74d-2f27-4665-8fd4-4ad8949bca4f None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:54.561630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f74d-2f27-4665-8fd4-4ad8949bca4f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:54.562053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f74d-2f27-4665-8fd4-4ad8949bca4f None admin] PATH_INFO: `/v3/projects/a9ddf9c6d32c459d8f135c30485e0f10/groups/34c9049ade024859804f8298dbfc8db3/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:54.564351 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7f74d-2f27-4665-8fd4-4ad8949bca4f None admin] RBAC: Authorizing `identity:create_grant(project_id=a9ddf9c6d32c459d8f135c30485e0f10, group_id=34c9049ade024859804f8298dbfc8db3, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:54.565992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7f74d-2f27-4665-8fd4-4ad8949bca4f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:54.589202 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 218/415] 10.222.0.70 () {58 vars in 1442 bytes} [Tue Jul 19 20:34:54 2022] PUT /identity/v3/projects/a9ddf9c6d32c459d8f135c30485e0f10/groups/34c9049ade024859804f8298dbfc8db3/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:34:54.939126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4c794-d16b-49a9-bc4b-44c0fdfbd82f None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:54.939648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4c794-d16b-49a9-bc4b-44c0fdfbd82f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:54.939757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4c794-d16b-49a9-bc4b-44c0fdfbd82f None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:54.941542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 198/416] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:54.947446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f66efb-209c-49b4-ac3c-a30f361e39f3 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:54.947772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f66efb-209c-49b4-ac3c-a30f361e39f3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:54.948014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f66efb-209c-49b4-ac3c-a30f361e39f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:54.973417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e4f66efb-209c-49b4-ac3c-a30f361e39f3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:54.974709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e4f66efb-209c-49b4-ac3c-a30f361e39f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:55.060961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 219/417] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:55.098924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4c794-d16b-49a9-bc4b-44c0fdfbd82f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:55.311895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4c794-d16b-49a9-bc4b-44c0fdfbd82f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:55.356632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:55.415806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74e51c-17f0-42e1-8537-5ab937c11493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:55.417234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74e51c-17f0-42e1-8537-5ab937c11493 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:55.417761 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74e51c-17f0-42e1-8537-5ab937c11493 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:55.418239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74e51c-17f0-42e1-8537-5ab937c11493 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:55.444246 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74e51c-17f0-42e1-8537-5ab937c11493 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-2b74e51c-17f0-42e1-8537-5ab937c11493 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:55.463137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:55.464826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:55.467006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 199/418] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:55.474927 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f66efb-209c-49b4-ac3c-a30f361e39f3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:55.558063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:55.615206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f66efb-209c-49b4-ac3c-a30f361e39f3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:55.663839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c41af-ee01-41da-8e27-79393d1fa4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:55.664537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c41af-ee01-41da-8e27-79393d1fa4f4 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:55.664901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c41af-ee01-41da-8e27-79393d1fa4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:55.665242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c41af-ee01-41da-8e27-79393d1fa4f4 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:55.666451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c41af-ee01-41da-8e27-79393d1fa4f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:55.666972 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c41af-ee01-41da-8e27-79393d1fa4f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:55.669102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c41af-ee01-41da-8e27-79393d1fa4f4 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:55.683238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 220/419] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:55.695395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74e51c-17f0-42e1-8537-5ab937c11493 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:55.932489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74e51c-17f0-42e1-8537-5ab937c11493 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:55.999261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ab5ba-ab4f-4b8e-96b1-bb36ee4f230b None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:55.999781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ab5ba-ab4f-4b8e-96b1-bb36ee4f230b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:56.000292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ab5ba-ab4f-4b8e-96b1-bb36ee4f230b None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:56.003344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 221/420] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:34:56.012920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b088b-9232-4ea6-9697-e4bf33d41f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:56.013639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa434a-e313-40ca-9240-a97a078d5cae None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:56.013925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa434a-e313-40ca-9240-a97a078d5cae None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:56.014071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b088b-9232-4ea6-9697-e4bf33d41f7d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:56.014365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa434a-e313-40ca-9240-a97a078d5cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:56.014507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b088b-9232-4ea6-9697-e4bf33d41f7d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:56.015008 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b088b-9232-4ea6-9697-e4bf33d41f7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:56.083203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-eaaa434a-e313-40ca-9240-a97a078d5cae None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:56.085259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-eaaa434a-e313-40ca-9240-a97a078d5cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:56.135164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:56.182923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b088b-9232-4ea6-9697-e4bf33d41f7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:56.187661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b088b-9232-4ea6-9697-e4bf33d41f7d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:56.302568 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 222/421] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:56.350449 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa434a-e313-40ca-9240-a97a078d5cae None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:56.462778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 200/422] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 768 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:56.472519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b088b-9232-4ea6-9697-e4bf33d41f7d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:56.522206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:56.628829 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa434a-e313-40ca-9240-a97a078d5cae None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:56.665114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:56.692179 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62557cf4-2405-4415-9df6-41dfa9e28200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:56.694720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62557cf4-2405-4415-9df6-41dfa9e28200 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:56.695532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62557cf4-2405-4415-9df6-41dfa9e28200 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:56.696534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62557cf4-2405-4415-9df6-41dfa9e28200 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:56.705495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b088b-9232-4ea6-9697-e4bf33d41f7d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:56.718186 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62557cf4-2405-4415-9df6-41dfa9e28200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-62557cf4-2405-4415-9df6-41dfa9e28200 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:56.738418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:56.739802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:56.741129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 223/423] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:34:56.749952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d142e1-188e-43e0-b6d7-65af03000c9b None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:56.749952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d142e1-188e-43e0-b6d7-65af03000c9b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:56.749952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d142e1-188e-43e0-b6d7-65af03000c9b None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:56.753939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 224/424] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:56.758346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9017f-3b15-4e78-88ba-bf087908d4b4 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:56.758900 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9017f-3b15-4e78-88ba-bf087908d4b4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:56.759402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9017f-3b15-4e78-88ba-bf087908d4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:56.788834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-aac9017f-3b15-4e78-88ba-bf087908d4b4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:56.791518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-aac9017f-3b15-4e78-88ba-bf087908d4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:56.791750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014ceda-a455-4fd3-ae58-002ede2631e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:56.793191 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ceda-a455-4fd3-ae58-002ede2631e4 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:56.794279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ceda-a455-4fd3-ae58-002ede2631e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:56.794779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ceda-a455-4fd3-ae58-002ede2631e4 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:56.797167 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014ceda-a455-4fd3-ae58-002ede2631e4 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:56.799937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014ceda-a455-4fd3-ae58-002ede2631e4 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:56.810106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 201/425] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:56 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:56.811913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014ceda-a455-4fd3-ae58-002ede2631e4 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:56.916411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 225/426] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:34:56.919854 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9017f-3b15-4e78-88ba-bf087908d4b4 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:57.007103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:57.105517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014ceda-a455-4fd3-ae58-002ede2631e4 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:57.133486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9017f-3b15-4e78-88ba-bf087908d4b4 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:57.175555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357af4f9-e8fd-49a6-bf31-d9767c54ec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:57.176682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357af4f9-e8fd-49a6-bf31-d9767c54ec4e None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:57.177186 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357af4f9-e8fd-49a6-bf31-d9767c54ec4e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:57.177724 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357af4f9-e8fd-49a6-bf31-d9767c54ec4e None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:57.179229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357af4f9-e8fd-49a6-bf31-d9767c54ec4e None admin] RBAC: Adding query filter params (name=member) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:57.179778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357af4f9-e8fd-49a6-bf31-d9767c54ec4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:57.182520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357af4f9-e8fd-49a6-bf31-d9767c54ec4e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:57.195560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc5256-7754-4df7-a552-9e21b155066a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:57.196981 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5256-7754-4df7-a552-9e21b155066a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:57.197460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5256-7754-4df7-a552-9e21b155066a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:57.197887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5256-7754-4df7-a552-9e21b155066a None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:57.199738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 202/427] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:34:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:57.200395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc5256-7754-4df7-a552-9e21b155066a None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:57.201452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357af4f9-e8fd-49a6-bf31-d9767c54ec4e None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:57.203146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc5256-7754-4df7-a552-9e21b155066a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:57.206566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 226/428] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:56 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:57.208796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8428f1e-0544-4289-96d3-7d12bc78f90c None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:57.209236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8428f1e-0544-4289-96d3-7d12bc78f90c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:57.209691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8428f1e-0544-4289-96d3-7d12bc78f90c None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:57.212260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 227/429] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:57.213505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8428f1e-0544-4289-96d3-7d12bc78f90c None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:57.231654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:57.381824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357af4f9-e8fd-49a6-bf31-d9767c54ec4e None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:57.384334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8428f1e-0544-4289-96d3-7d12bc78f90c None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:57.407850 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:57.435029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1e306-3d78-4df8-a470-785c957d7f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:57.436696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e306-3d78-4df8-a470-785c957d7f92 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:57.437155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e306-3d78-4df8-a470-785c957d7f92 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:57.437635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e306-3d78-4df8-a470-785c957d7f92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:57.450769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ca02-87cd-4521-8698-569b2ac586d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:57.451454 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ca02-87cd-4521-8698-569b2ac586d3 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:57.451733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ca02-87cd-4521-8698-569b2ac586d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:57.452012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ca02-87cd-4521-8698-569b2ac586d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:57.459832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1e306-3d78-4df8-a470-785c957d7f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-09c1e306-3d78-4df8-a470-785c957d7f92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:57.477017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:57.478829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:34:57.480257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 203/430] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:57.483817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1e306-3d78-4df8-a470-785c957d7f92 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:57.543622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8ca02-87cd-4521-8698-569b2ac586d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:57.545144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8ca02-87cd-4521-8698-569b2ac586d3 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:57.627711 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:57.666237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1e306-3d78-4df8-a470-785c957d7f92 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:57.714900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89f8e8-e163-47a2-9a15-e077980d745f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:57.716375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89f8e8-e163-47a2-9a15-e077980d745f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:57.716884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89f8e8-e163-47a2-9a15-e077980d745f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:57.717431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89f8e8-e163-47a2-9a15-e077980d745f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:57.718430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89f8e8-e163-47a2-9a15-e077980d745f None admin] RBAC: Adding query filter params (group.id=7cde2658873347688bffa512ef1b112e, role.id=d94de411ec0a40be92c04950e6edcac7, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:57.718757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89f8e8-e163-47a2-9a15-e077980d745f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:57.720286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89f8e8-e163-47a2-9a15-e077980d745f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:57.727598 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 204/431] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jul 19 20:34:57 2022] GET /identity/v3/role_assignments?group.id=7cde2658873347688bffa512ef1b112e&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 611 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:57.728948 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76e05b-67e3-4505-9774-4cf1e123c9c2 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:57.729239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76e05b-67e3-4505-9774-4cf1e123c9c2 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:57.729565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76e05b-67e3-4505-9774-4cf1e123c9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:57.742327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 228/432] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:57.744608 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ca02-87cd-4521-8698-569b2ac586d3 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:57.763112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-5b76e05b-67e3-4505-9774-4cf1e123c9c2 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:57.764371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5b76e05b-67e3-4505-9774-4cf1e123c9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:57.769816 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:57.900476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 205/433] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:57.902288 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76e05b-67e3-4505-9774-4cf1e123c9c2 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:57.971332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ca02-87cd-4521-8698-569b2ac586d3 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:58.030432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82358d9-e14c-4524-9f87-4aa12aade605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:58.031124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82358d9-e14c-4524-9f87-4aa12aade605 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:58.031408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82358d9-e14c-4524-9f87-4aa12aade605 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:58.031681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82358d9-e14c-4524-9f87-4aa12aade605 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:58.032446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82358d9-e14c-4524-9f87-4aa12aade605 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:58.032857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82358d9-e14c-4524-9f87-4aa12aade605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:58.035161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82358d9-e14c-4524-9f87-4aa12aade605 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:58.048064 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 229/434] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:58.048946 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82358d9-e14c-4524-9f87-4aa12aade605 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:58.056251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:58.130899 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76e05b-67e3-4505-9774-4cf1e123c9c2 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:58.200903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb8553-6c57-412c-91ea-398ef7a67b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:58.202986 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb8553-6c57-412c-91ea-398ef7a67b56 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:58.203797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb8553-6c57-412c-91ea-398ef7a67b56 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:58.204313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb8553-6c57-412c-91ea-398ef7a67b56 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:58.206774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb8553-6c57-412c-91ea-398ef7a67b56 None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:58.209632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb8553-6c57-412c-91ea-398ef7a67b56 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:58.211240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:58.224997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 206/435] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:57 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:58.224997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb8553-6c57-412c-91ea-398ef7a67b56 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:58.288794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82358d9-e14c-4524-9f87-4aa12aade605 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:58.357637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7634b1-7f8a-45ad-a860-462b0861151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:58.358653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7634b1-7f8a-45ad-a860-462b0861151d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:58.358949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7634b1-7f8a-45ad-a860-462b0861151d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:58.359240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7634b1-7f8a-45ad-a860-462b0861151d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:58.378560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7634b1-7f8a-45ad-a860-462b0861151d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-7a7634b1-7f8a-45ad-a860-462b0861151d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:58.391173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:58.393160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 230/436] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:34:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:58.394955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7634b1-7f8a-45ad-a860-462b0861151d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:58.434825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb8553-6c57-412c-91ea-398ef7a67b56 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:58.476386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:58.486329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cb779-96e9-4f5b-8bfe-739a5a5b38df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:58.487428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb779-96e9-4f5b-8bfe-739a5a5b38df None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:58.487893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb779-96e9-4f5b-8bfe-739a5a5b38df None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:58.488346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb779-96e9-4f5b-8bfe-739a5a5b38df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:58.560862 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7634b1-7f8a-45ad-a860-462b0861151d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:58.603749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cb779-96e9-4f5b-8bfe-739a5a5b38df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:58.605660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cb779-96e9-4f5b-8bfe-739a5a5b38df None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:58.611842 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:58.620214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb2929-5360-49b8-901f-497353a40766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:58.621326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb2929-5360-49b8-901f-497353a40766 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:58.622379 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb2929-5360-49b8-901f-497353a40766 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:58.622832 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb2929-5360-49b8-901f-497353a40766 None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:58.624853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb2929-5360-49b8-901f-497353a40766 None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:58.627288 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb2929-5360-49b8-901f-497353a40766 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:58.630214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 231/437] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:58 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:58.632060 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4683d-27fe-48a7-a43d-7909c98a81b1 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:58.632382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4683d-27fe-48a7-a43d-7909c98a81b1 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:58.632692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4683d-27fe-48a7-a43d-7909c98a81b1 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:58.634201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 232/438] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:34:58.635505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4683d-27fe-48a7-a43d-7909c98a81b1 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:58.763244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 207/439] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:58.764331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cb779-96e9-4f5b-8bfe-739a5a5b38df None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:58.808656 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4683d-27fe-48a7-a43d-7909c98a81b1 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:58.821430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:58.853278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95804ae4-ff16-47f5-861b-038e984a88ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:58.854020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95804ae4-ff16-47f5-861b-038e984a88ea None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:58.854302 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95804ae4-ff16-47f5-861b-038e984a88ea None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:58.854574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95804ae4-ff16-47f5-861b-038e984a88ea None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:58.855350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95804ae4-ff16-47f5-861b-038e984a88ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:58.855654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95804ae4-ff16-47f5-861b-038e984a88ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:58.857692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95804ae4-ff16-47f5-861b-038e984a88ea None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:58.868081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 233/440] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:34:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:58.869576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d27000-69e5-4dfd-a5ec-8066a7631891 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:58.869932 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d27000-69e5-4dfd-a5ec-8066a7631891 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:58.870217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d27000-69e5-4dfd-a5ec-8066a7631891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:58.888550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-62d27000-69e5-4dfd-a5ec-8066a7631891 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:34:58.889810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-62d27000-69e5-4dfd-a5ec-8066a7631891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:34:58.945705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cb779-96e9-4f5b-8bfe-739a5a5b38df None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:58.980355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:58.992777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae828e9e-8aed-4717-9518-26d2f4d284f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:58.993992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae828e9e-8aed-4717-9518-26d2f4d284f2 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:58.994487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae828e9e-8aed-4717-9518-26d2f4d284f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:58.994974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae828e9e-8aed-4717-9518-26d2f4d284f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:58.996524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae828e9e-8aed-4717-9518-26d2f4d284f2 None admin] RBAC: Adding query filter params (group.id=7cde2658873347688bffa512ef1b112e, role.id=d94de411ec0a40be92c04950e6edcac7, scope.project.id=0d8860ce338747a5ba3d5903b48cc986) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:58.996899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 234/441] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:34:58.997077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae828e9e-8aed-4717-9518-26d2f4d284f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:58.998214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d27000-69e5-4dfd-a5ec-8066a7631891 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:58.999833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae828e9e-8aed-4717-9518-26d2f4d284f2 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:59.015431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 208/442] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jul 19 20:34:58 2022] GET /identity/v3/role_assignments?group.id=7cde2658873347688bffa512ef1b112e&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=d94de411ec0a40be92c04950e6edcac7 => generated 611 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:59.015431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae828e9e-8aed-4717-9518-26d2f4d284f2 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:59.154682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:59.169685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae828e9e-8aed-4717-9518-26d2f4d284f2 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:59.200413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d27000-69e5-4dfd-a5ec-8066a7631891 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:59.234686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d604a-5295-479a-ae14-3569423812c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:59.235723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d604a-5295-479a-ae14-3569423812c2 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:59.236173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d604a-5295-479a-ae14-3569423812c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:59.236617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d604a-5295-479a-ae14-3569423812c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:59.269996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b852b-6649-414c-a821-ea9f8e656f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:59.271024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b852b-6649-414c-a821-ea9f8e656f65 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:59.271420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b852b-6649-414c-a821-ea9f8e656f65 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:59.271815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b852b-6649-414c-a821-ea9f8e656f65 None admin] PATH_INFO: `/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:59.274982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b852b-6649-414c-a821-ea9f8e656f65 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9a5a71e45c4bd1b4dcbd618b5109dc)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:59.280502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b852b-6649-414c-a821-ea9f8e656f65 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:59.280502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 235/443] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:58 2022] GET /identity/v3/projects/ea9a5a71e45c4bd1b4dcbd618b5109dc => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:59.281994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b852b-6649-414c-a821-ea9f8e656f65 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:59.370778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d604a-5295-479a-ae14-3569423812c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:59.373643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d604a-5295-479a-ae14-3569423812c2 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:59.395767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:59.452598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b852b-6649-414c-a821-ea9f8e656f65 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:59.504438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52142653-9ef0-4079-beef-e4a881ddeaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:59.505255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52142653-9ef0-4079-beef-e4a881ddeaba None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:59.505557 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52142653-9ef0-4079-beef-e4a881ddeaba None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:59.505844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52142653-9ef0-4079-beef-e4a881ddeaba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:59.518035 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52142653-9ef0-4079-beef-e4a881ddeaba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:34:59.525470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-52142653-9ef0-4079-beef-e4a881ddeaba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:34:59.526243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:34:59.527746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 236/444] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:34:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:34:59.529171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52142653-9ef0-4079-beef-e4a881ddeaba None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:59.544079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 209/445] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:34:59.545509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d604a-5295-479a-ae14-3569423812c2 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:59.675419 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d604a-5295-479a-ae14-3569423812c2 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:59.687365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52142653-9ef0-4079-beef-e4a881ddeaba None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:59.712751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:59.731213 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572e281-764b-45af-a99c-47e196fd4896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:59.732214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572e281-764b-45af-a99c-47e196fd4896 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:59.732661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572e281-764b-45af-a99c-47e196fd4896 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:59.732993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572e281-764b-45af-a99c-47e196fd4896 None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:59.734519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572e281-764b-45af-a99c-47e196fd4896 None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:59.735122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4cf82-2096-4918-ae76-4964daca54cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:59.736181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572e281-764b-45af-a99c-47e196fd4896 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:59.736349 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4cf82-2096-4918-ae76-4964daca54cc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:59.736622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4cf82-2096-4918-ae76-4964daca54cc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:59.737061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4cf82-2096-4918-ae76-4964daca54cc None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:59.738628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 237/446] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:59 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:59.738824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4cf82-2096-4918-ae76-4964daca54cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:59.738824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4cf82-2096-4918-ae76-4964daca54cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:59.739461 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572e281-764b-45af-a99c-47e196fd4896 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:59.741196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4cf82-2096-4918-ae76-4964daca54cc None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:59.759386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 210/447] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:34:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:59.760619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4cf82-2096-4918-ae76-4964daca54cc None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:59.871250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572e281-764b-45af-a99c-47e196fd4896 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:59.884959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:34:59.920881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5fe4e-8ab6-4623-92f8-943a41fe7ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:59.922258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5fe4e-8ab6-4623-92f8-943a41fe7ed3 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:59.922712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5fe4e-8ab6-4623-92f8-943a41fe7ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:59.923166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5fe4e-8ab6-4623-92f8-943a41fe7ed3 None admin] PATH_INFO: `/v3/projects/0d8860ce338747a5ba3d5903b48cc986` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:59.925124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5fe4e-8ab6-4623-92f8-943a41fe7ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=0d8860ce338747a5ba3d5903b48cc986)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:59.927622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5fe4e-8ab6-4623-92f8-943a41fe7ed3 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:59.930839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 238/448] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:34:59 2022] GET /identity/v3/projects/0d8860ce338747a5ba3d5903b48cc986 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:59.931942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5fe4e-8ab6-4623-92f8-943a41fe7ed3 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:34:59.947894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4cf82-2096-4918-ae76-4964daca54cc None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:34:59.987748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b2231-04aa-45b1-a79e-cd3d78a698dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:34:59.988411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b2231-04aa-45b1-a79e-cd3d78a698dc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:34:59.988693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b2231-04aa-45b1-a79e-cd3d78a698dc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:34:59.989473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b2231-04aa-45b1-a79e-cd3d78a698dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:34:59.989907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b2231-04aa-45b1-a79e-cd3d78a698dc None admin] RBAC: Adding query filter params (group.id=7cde2658873347688bffa512ef1b112e, role.id=0044294fcceb4b6ba0cb27567f2761d4, scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:34:59.990252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b2231-04aa-45b1-a79e-cd3d78a698dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:34:59.991729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b2231-04aa-45b1-a79e-cd3d78a698dc None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:34:59.998977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 211/449] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jul 19 20:34:59 2022] GET /identity/v3/role_assignments?group.id=7cde2658873347688bffa512ef1b112e&scope.project.id=ea9a5a71e45c4bd1b4dcbd618b5109dc&role.id=0044294fcceb4b6ba0cb27567f2761d4 => generated 611 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:34:59.999814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b2231-04aa-45b1-a79e-cd3d78a698dc None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:00.052369 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:00.116486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5fe4e-8ab6-4623-92f8-943a41fe7ed3 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:00.166007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef28df-6192-404a-8942-9647924a2152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:00.167268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef28df-6192-404a-8942-9647924a2152 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:00.167732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef28df-6192-404a-8942-9647924a2152 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:00.168181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef28df-6192-404a-8942-9647924a2152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:00.175316 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b2231-04aa-45b1-a79e-cd3d78a698dc None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:00.218406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:00.241944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f2d5c-6454-4993-b98a-f7bac5767d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:00.243458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f2d5c-6454-4993-b98a-f7bac5767d15 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:00.243967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f2d5c-6454-4993-b98a-f7bac5767d15 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:00.244471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f2d5c-6454-4993-b98a-f7bac5767d15 None admin] PATH_INFO: `/v3/groups/7cde2658873347688bffa512ef1b112e` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:00.246753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f2d5c-6454-4993-b98a-f7bac5767d15 None admin] RBAC: Authorizing `identity:get_group(group_id=7cde2658873347688bffa512ef1b112e)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:00.249632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f2d5c-6454-4993-b98a-f7bac5767d15 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:00.253109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 212/450] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:34:59 2022] GET /identity/v3/groups/7cde2658873347688bffa512ef1b112e => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:00.262626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f2d5c-6454-4993-b98a-f7bac5767d15 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:00.312483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef28df-6192-404a-8942-9647924a2152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:00.315471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef28df-6192-404a-8942-9647924a2152 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:00.414784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:00.443166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f2d5c-6454-4993-b98a-f7bac5767d15 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:00.502979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 239/451] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:34:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:00.504430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5cd7-64b2-4f9f-8246-2a1a47293a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:00.505636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5cd7-64b2-4f9f-8246-2a1a47293a68 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:00.506156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5cd7-64b2-4f9f-8246-2a1a47293a68 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:00.506666 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5cd7-64b2-4f9f-8246-2a1a47293a68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:00.508391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5cd7-64b2-4f9f-8246-2a1a47293a68 None admin] RBAC: Adding query filter params (group.id=7cde2658873347688bffa512ef1b112e, role.id=0044294fcceb4b6ba0cb27567f2761d4, scope.project.id=0d8860ce338747a5ba3d5903b48cc986) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:00.508970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5cd7-64b2-4f9f-8246-2a1a47293a68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:00.513386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5cd7-64b2-4f9f-8246-2a1a47293a68 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:00.513529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef28df-6192-404a-8942-9647924a2152 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:00.523299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 213/452] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jul 19 20:35:00 2022] GET /identity/v3/role_assignments?group.id=7cde2658873347688bffa512ef1b112e&scope.project.id=0d8860ce338747a5ba3d5903b48cc986&role.id=0044294fcceb4b6ba0cb27567f2761d4 => generated 611 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:00.695913 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef28df-6192-404a-8942-9647924a2152 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:00.751247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847b367-e6d7-4867-8264-7fb52a836e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:00.752820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847b367-e6d7-4867-8264-7fb52a836e71 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:00.753343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847b367-e6d7-4867-8264-7fb52a836e71 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:00.753936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847b367-e6d7-4867-8264-7fb52a836e71 None admin] PATH_INFO: `/v3/projects/a9ddf9c6d32c459d8f135c30485e0f10` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:00.756232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847b367-e6d7-4867-8264-7fb52a836e71 None admin] RBAC: Authorizing `identity:get_project(project_id=a9ddf9c6d32c459d8f135c30485e0f10)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:00.759039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847b367-e6d7-4867-8264-7fb52a836e71 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:00.763109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 240/453] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:35:00 2022] GET /identity/v3/projects/a9ddf9c6d32c459d8f135c30485e0f10 => generated 344 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:00.769749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5cd7-64b2-4f9f-8246-2a1a47293a68 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:00.843010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:00.918864 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5cd7-64b2-4f9f-8246-2a1a47293a68 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:00.965255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f6ca7-328b-4a67-912c-35f9d4413d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:00.966151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f6ca7-328b-4a67-912c-35f9d4413d36 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:00.966475 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f6ca7-328b-4a67-912c-35f9d4413d36 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:00.966794 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f6ca7-328b-4a67-912c-35f9d4413d36 None admin] PATH_INFO: `/v3/groups/34c9049ade024859804f8298dbfc8db3` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:00.968061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f6ca7-328b-4a67-912c-35f9d4413d36 None admin] RBAC: Authorizing `identity:get_group(group_id=34c9049ade024859804f8298dbfc8db3)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:00.969583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f6ca7-328b-4a67-912c-35f9d4413d36 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:00.971280 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 214/454] 10.222.0.70 () {56 vars in 1180 bytes} [Tue Jul 19 20:35:00 2022] GET /identity/v3/groups/34c9049ade024859804f8298dbfc8db3 => generated 229 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:00.978707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847b367-e6d7-4867-8264-7fb52a836e71 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:01.005269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:01.084875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847b367-e6d7-4867-8264-7fb52a836e71 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:01.119602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d08db7-ba60-4d95-b8a0-44663a3d31d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:01.120248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d08db7-ba60-4d95-b8a0-44663a3d31d4 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:01.120529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d08db7-ba60-4d95-b8a0-44663a3d31d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:01.120803 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d08db7-ba60-4d95-b8a0-44663a3d31d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:01.122173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d08db7-ba60-4d95-b8a0-44663a3d31d4 None admin] RBAC: Adding query filter params (group.id=34c9049ade024859804f8298dbfc8db3, role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=a9ddf9c6d32c459d8f135c30485e0f10) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:01.122479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d08db7-ba60-4d95-b8a0-44663a3d31d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:01.123904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d08db7-ba60-4d95-b8a0-44663a3d31d4 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:01.129738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 241/455] 10.222.0.70 () {56 vars in 1376 bytes} [Tue Jul 19 20:35:00 2022] GET /identity/v3/role_assignments?group.id=34c9049ade024859804f8298dbfc8db3&scope.project.id=a9ddf9c6d32c459d8f135c30485e0f10&role.id=85420c68300642aabfd730fb8fef8347 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:04.578432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4eeb3-e10b-4858-bce3-79463455fe5b None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:04.579953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4eeb3-e10b-4858-bce3-79463455fe5b None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:04.580724 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4eeb3-e10b-4858-bce3-79463455fe5b None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:04.584142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 215/456] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:04.589397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee21a96-cb5f-423a-91cd-988985b7c856 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:04.590421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee21a96-cb5f-423a-91cd-988985b7c856 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:04.590883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee21a96-cb5f-423a-91cd-988985b7c856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:04.623601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0ee21a96-cb5f-423a-91cd-988985b7c856 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:04.625667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0ee21a96-cb5f-423a-91cd-988985b7c856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:04.793294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 242/457] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:35:04.850128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4eeb3-e10b-4858-bce3-79463455fe5b None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:04.924044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac37a0-9046-4e51-95d0-24834f2493a1 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:04.924465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac37a0-9046-4e51-95d0-24834f2493a1 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:04.924959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac37a0-9046-4e51-95d0-24834f2493a1 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:04.927527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 243/458] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:04.935131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85bdc-903e-4e4e-af85-25c3a954d94a None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:04.935632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85bdc-903e-4e4e-af85-25c3a954d94a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:04.936126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85bdc-903e-4e4e-af85-25c3a954d94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:04.971410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-51c85bdc-903e-4e4e-af85-25c3a954d94a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:04.973305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-51c85bdc-903e-4e4e-af85-25c3a954d94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:04.991930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:05.050928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 244/459] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:35:05.052404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16a33e-3a1c-47a0-b4b5-b0b851455d9f None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:05.052701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16a33e-3a1c-47a0-b4b5-b0b851455d9f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:05.052988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16a33e-3a1c-47a0-b4b5-b0b851455d9f None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:05.055132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 245/460] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:05.067035 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05abb6-5b9c-4dd4-9902-9094fdf30046 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:05.067336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05abb6-5b9c-4dd4-9902-9094fdf30046 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:05.067614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05abb6-5b9c-4dd4-9902-9094fdf30046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:05.082492 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4eeb3-e10b-4858-bce3-79463455fe5b None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:05.085720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-6e05abb6-5b9c-4dd4-9902-9094fdf30046 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:05.087329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6e05abb6-5b9c-4dd4-9902-9094fdf30046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:05.125086 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:05.144834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637696d7-5a1d-46c8-bbd4-afa7bbaa9560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:05.146324 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637696d7-5a1d-46c8-bbd4-afa7bbaa9560 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:05.146900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637696d7-5a1d-46c8-bbd4-afa7bbaa9560 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:05.147464 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637696d7-5a1d-46c8-bbd4-afa7bbaa9560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:05.164864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 246/461] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:05.165786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05abb6-5b9c-4dd4-9902-9094fdf30046 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:05.168855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637696d7-5a1d-46c8-bbd4-afa7bbaa9560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:05.171721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637696d7-5a1d-46c8-bbd4-afa7bbaa9560 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-637696d7-5a1d-46c8-bbd4-afa7bbaa9560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:05.205426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:05.206802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:05.207727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:05.207727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:05.207727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:05.207727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:05.208061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 216/462] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:05.218108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637696d7-5a1d-46c8-bbd4-afa7bbaa9560 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:05.257600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:05.281725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05abb6-5b9c-4dd4-9902-9094fdf30046 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:05.320267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63075134-8c90-406c-b040-019eb8bc98cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:05.321074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63075134-8c90-406c-b040-019eb8bc98cd None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:05.321396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63075134-8c90-406c-b040-019eb8bc98cd None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:05.321644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63075134-8c90-406c-b040-019eb8bc98cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:05.332987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63075134-8c90-406c-b040-019eb8bc98cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:05.334653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63075134-8c90-406c-b040-019eb8bc98cd None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-63075134-8c90-406c-b040-019eb8bc98cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:05.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:05.358128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:05.361687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:05.361687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:05.361687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:05.361687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:05.361687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 247/463] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:05.361687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63075134-8c90-406c-b040-019eb8bc98cd None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:05.431505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637696d7-5a1d-46c8-bbd4-afa7bbaa9560 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:05.490415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bdcab-cbc6-4917-842b-79ec046166dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:05.491776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bdcab-cbc6-4917-842b-79ec046166dc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:05.492428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bdcab-cbc6-4917-842b-79ec046166dc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:05.493095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bdcab-cbc6-4917-842b-79ec046166dc None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:05.494577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bdcab-cbc6-4917-842b-79ec046166dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:05.495291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bdcab-cbc6-4917-842b-79ec046166dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:05.498001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bdcab-cbc6-4917-842b-79ec046166dc None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:05.509852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63075134-8c90-406c-b040-019eb8bc98cd None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:05.531036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 217/464] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:05.532913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bdcab-cbc6-4917-842b-79ec046166dc None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:05.563259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:05.574978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124e697-9114-4269-b390-548056498ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:05.576190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124e697-9114-4269-b390-548056498ae1 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:05.577160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124e697-9114-4269-b390-548056498ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:05.577314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124e697-9114-4269-b390-548056498ae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:05.591973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:05.599861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124e697-9114-4269-b390-548056498ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:05.602397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124e697-9114-4269-b390-548056498ae1 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-9124e697-9114-4269-b390-548056498ae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:05.634888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:05.637821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:05.639980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:05.639980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:05.639980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:05.639980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:05.639980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 248/465] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:05.639980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124e697-9114-4269-b390-548056498ae1 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:05.712229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bdcab-cbc6-4917-842b-79ec046166dc None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:05.772631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf568a4-bd91-429d-9a3e-a98e9a10dabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:05.773640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf568a4-bd91-429d-9a3e-a98e9a10dabc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:05.774062 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf568a4-bd91-429d-9a3e-a98e9a10dabc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:05.774528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf568a4-bd91-429d-9a3e-a98e9a10dabc None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:05.775394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf568a4-bd91-429d-9a3e-a98e9a10dabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:05.775762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf568a4-bd91-429d-9a3e-a98e9a10dabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:05.778346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf568a4-bd91-429d-9a3e-a98e9a10dabc None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:05.801944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 218/466] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:05.801944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21c4f5-c03f-4111-9798-b5334fc08b1c None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:05.801944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21c4f5-c03f-4111-9798-b5334fc08b1c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:05.801944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21c4f5-c03f-4111-9798-b5334fc08b1c None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:05.801944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 219/467] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:05.803341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21c4f5-c03f-4111-9798-b5334fc08b1c None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:05.830152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124e697-9114-4269-b390-548056498ae1 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:05.870165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde8326-9e90-4315-8bda-2125f0b570cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:05.871881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8326-9e90-4315-8bda-2125f0b570cf None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:05.871881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8326-9e90-4315-8bda-2125f0b570cf None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:05.872277 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8326-9e90-4315-8bda-2125f0b570cf None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:05.872945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde8326-9e90-4315-8bda-2125f0b570cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:05.874655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde8326-9e90-4315-8bda-2125f0b570cf None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:05.908240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 249/468] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:35:05 2022] POST /identity/v3/users => generated 246 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:05.909257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde8326-9e90-4315-8bda-2125f0b570cf None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:05.991810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:05.996093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:06.013879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21c4f5-c03f-4111-9798-b5334fc08b1c None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:06.095313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2158fa3-82d9-4055-801f-005c74671b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:06.096380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2158fa3-82d9-4055-801f-005c74671b57 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:06.096846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2158fa3-82d9-4055-801f-005c74671b57 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:06.097301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2158fa3-82d9-4055-801f-005c74671b57 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:06.098508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2158fa3-82d9-4055-801f-005c74671b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:06.099021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2158fa3-82d9-4055-801f-005c74671b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:06.101305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2158fa3-82d9-4055-801f-005c74671b57 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:06.116777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde8326-9e90-4315-8bda-2125f0b570cf None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:06.125937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 220/469] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:06.128033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6c546-a389-42cb-adbf-d61a8ee2a6db None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:06.128505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6c546-a389-42cb-adbf-d61a8ee2a6db None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:06.129128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6c546-a389-42cb-adbf-d61a8ee2a6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:06.147895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c8f6c546-a389-42cb-adbf-d61a8ee2a6db None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:06.149135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c8f6c546-a389-42cb-adbf-d61a8ee2a6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:06.185520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7f542-8631-4e35-bc05-580a416fe599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:06.186977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7f542-8631-4e35-bc05-580a416fe599 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:06.187687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7f542-8631-4e35-bc05-580a416fe599 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:06.188180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7f542-8631-4e35-bc05-580a416fe599 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:06.189580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7f542-8631-4e35-bc05-580a416fe599 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:06.192294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7f542-8631-4e35-bc05-580a416fe599 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:06.233392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 250/470] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:35:05 2022] POST /identity/v3/users => generated 248 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:06.233392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e080f7b-c57d-4db5-9fff-016b6e38ae32 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:06.233392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e080f7b-c57d-4db5-9fff-016b6e38ae32 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:06.233392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e080f7b-c57d-4db5-9fff-016b6e38ae32 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:06.233392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 251/471] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:06.233392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e080f7b-c57d-4db5-9fff-016b6e38ae32 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:06.268035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 221/472] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:06.271252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5cd89-0b37-489a-8b5c-4e2b842a9e18 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:06.271767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5cd89-0b37-489a-8b5c-4e2b842a9e18 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:06.272674 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5cd89-0b37-489a-8b5c-4e2b842a9e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:06.308297 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-4ae5cd89-0b37-489a-8b5c-4e2b842a9e18 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:06.311633 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4ae5cd89-0b37-489a-8b5c-4e2b842a9e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:06.455322 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:06.475930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 222/473] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:06.475930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5cd89-0b37-489a-8b5c-4e2b842a9e18 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:06.476239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e080f7b-c57d-4db5-9fff-016b6e38ae32 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:06.496078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:06.545394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf01fd-8bd3-47b8-8027-f154068baff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:06.546270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf01fd-8bd3-47b8-8027-f154068baff2 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:06.546749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf01fd-8bd3-47b8-8027-f154068baff2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:06.547203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf01fd-8bd3-47b8-8027-f154068baff2 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:06.548498 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf01fd-8bd3-47b8-8027-f154068baff2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:06.551085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf01fd-8bd3-47b8-8027-f154068baff2 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:06.598936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 252/474] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:35:06 2022] POST /identity/v3/users => generated 248 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:06.601838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf01fd-8bd3-47b8-8027-f154068baff2 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:06.648876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5cd89-0b37-489a-8b5c-4e2b842a9e18 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:06.721432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df704-4d38-4315-95e5-ea6e8054c99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:06.721432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df704-4d38-4315-95e5-ea6e8054c99e None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:06.721432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df704-4d38-4315-95e5-ea6e8054c99e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:06.721432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df704-4d38-4315-95e5-ea6e8054c99e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:06.744689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524df704-4d38-4315-95e5-ea6e8054c99e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:06.748003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524df704-4d38-4315-95e5-ea6e8054c99e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:06.795862 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf01fd-8bd3-47b8-8027-f154068baff2 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-524df704-4d38-4315-95e5-ea6e8054c99e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:06.797531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:06.798517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:06.799959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:06.799959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:06.799959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:06.799959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:06.805123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 223/475] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:06.827630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df704-4d38-4315-95e5-ea6e8054c99e None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:06.883089 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3dd7e-98fc-498c-abc4-44247e09dc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:06.884547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3dd7e-98fc-498c-abc4-44247e09dc26 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:06.885259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3dd7e-98fc-498c-abc4-44247e09dc26 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:06.885785 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3dd7e-98fc-498c-abc4-44247e09dc26 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:06.886768 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3dd7e-98fc-498c-abc4-44247e09dc26 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:06.889344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3dd7e-98fc-498c-abc4-44247e09dc26 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:06.901547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.sql.core [None req-8bc3dd7e-98fc-498c-abc4-44247e09dc26 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 20:35:06.901547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 19 20:35:06.901547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [parameters: {'id': 'b46a0e4a2e604140bafd5771c9b1727a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 19 20:35:06.901547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92310) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-8bc3dd7e-98fc-498c-abc4-44247e09dc26 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application result.read() Jul 19 20:35:06.925094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return ref.to_dict() Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:35:06.926260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application yield resource Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise self.value Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application session.commit() Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 20:35:06.927460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self.session.flush() Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application compat.raise_( Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 20:35:06.928506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:35:06.929602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application result.read() Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:35:06.930611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [parameters: {'id': 'b46a0e4a2e604140bafd5771c9b1727a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:06.931802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 19 20:35:06.932899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:06.936641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 253/476] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Jul 19 20:35:06 2022] POST /identity/v3/roles => generated 144 bytes in 337 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:06.945537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3dd7e-98fc-498c-abc4-44247e09dc26 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:06.949268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:06.971616 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:07.025173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df704-4d38-4315-95e5-ea6e8054c99e None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:07.086938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3dd7e-98fc-498c-abc4-44247e09dc26 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:07.087924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe09d5-d21b-440e-aa9a-3b8ed4eedc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:07.089249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe09d5-d21b-440e-aa9a-3b8ed4eedc72 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:07.089953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe09d5-d21b-440e-aa9a-3b8ed4eedc72 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:07.090780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe09d5-d21b-440e-aa9a-3b8ed4eedc72 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:07.092400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe09d5-d21b-440e-aa9a-3b8ed4eedc72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:07.093217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe09d5-d21b-440e-aa9a-3b8ed4eedc72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:07.096154 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe09d5-d21b-440e-aa9a-3b8ed4eedc72 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:07.118292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 224/477] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:07.127368 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe09d5-d21b-440e-aa9a-3b8ed4eedc72 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:07.134564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d25411-9790-4a79-b89a-c5cc174da445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:07.136636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25411-9790-4a79-b89a-c5cc174da445 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:07.136636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25411-9790-4a79-b89a-c5cc174da445 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:07.136636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d25411-9790-4a79-b89a-c5cc174da445 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:07.148556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d25411-9790-4a79-b89a-c5cc174da445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-57d25411-9790-4a79-b89a-c5cc174da445 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:07.160201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:07.162006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:07.163700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 254/478] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:35:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:07.172536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d25411-9790-4a79-b89a-c5cc174da445 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:07.300221 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:07.342586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe09d5-d21b-440e-aa9a-3b8ed4eedc72 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:07.363652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d25411-9790-4a79-b89a-c5cc174da445 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:07.414947 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:07.429337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8dcd58-d59a-411b-9ead-53790e856ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:07.430530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dcd58-d59a-411b-9ead-53790e856ae0 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:07.431043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dcd58-d59a-411b-9ead-53790e856ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:07.431549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8dcd58-d59a-411b-9ead-53790e856ae0 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:07.432547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8dcd58-d59a-411b-9ead-53790e856ae0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:07.435828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8dcd58-d59a-411b-9ead-53790e856ae0 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:07.465226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0b600-81ec-420a-81a2-737aaa61cbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:07.466287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0b600-81ec-420a-81a2-737aaa61cbc7 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:07.466778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0b600-81ec-420a-81a2-737aaa61cbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:07.467151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0b600-81ec-420a-81a2-737aaa61cbc7 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:07.468117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0b600-81ec-420a-81a2-737aaa61cbc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:07.468430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0b600-81ec-420a-81a2-737aaa61cbc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:07.475296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 225/479] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:35:07 2022] POST /identity/v3/users => generated 249 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:07.475930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0b600-81ec-420a-81a2-737aaa61cbc7 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:07.498404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 255/480] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:08.923354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29082bd-c051-4615-a8a5-ebca80c33032 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:08.924164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29082bd-c051-4615-a8a5-ebca80c33032 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:08.924164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29082bd-c051-4615-a8a5-ebca80c33032 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:08.926186 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 226/481] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:08.932098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd7592-9a76-4f73-9d4f-e7ee151ce752 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:08.932863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd7592-9a76-4f73-9d4f-e7ee151ce752 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:08.932863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd7592-9a76-4f73-9d4f-e7ee151ce752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:08.959030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-d8cd7592-9a76-4f73-9d4f-e7ee151ce752 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:08.960255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d8cd7592-9a76-4f73-9d4f-e7ee151ce752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:09.067385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 256/482] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:09.110010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29082bd-c051-4615-a8a5-ebca80c33032 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:09.270387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29082bd-c051-4615-a8a5-ebca80c33032 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:09.321329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e1ea4-ab6d-4eba-9a7f-8a8cd0c740d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:09.322299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e1ea4-ab6d-4eba-9a7f-8a8cd0c740d0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:09.322586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e1ea4-ab6d-4eba-9a7f-8a8cd0c740d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:09.322966 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e1ea4-ab6d-4eba-9a7f-8a8cd0c740d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:09.342734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e1ea4-ab6d-4eba-9a7f-8a8cd0c740d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-4d1e1ea4-ab6d-4eba-9a7f-8a8cd0c740d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:09.356756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:09.357768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:09.358646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:09.359868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 227/483] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:09.366828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd7592-9a76-4f73-9d4f-e7ee151ce752 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:09.520335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:09.574221 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd7592-9a76-4f73-9d4f-e7ee151ce752 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:09.637397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c374dd-5abb-455a-9cd7-31c736f1fbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:09.638470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c374dd-5abb-455a-9cd7-31c736f1fbc3 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:09.638938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c374dd-5abb-455a-9cd7-31c736f1fbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:09.639403 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c374dd-5abb-455a-9cd7-31c736f1fbc3 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:09.640696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c374dd-5abb-455a-9cd7-31c736f1fbc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:09.641207 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c374dd-5abb-455a-9cd7-31c736f1fbc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:09.643537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c374dd-5abb-455a-9cd7-31c736f1fbc3 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:09.664244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 257/484] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:09.670396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e1ea4-ab6d-4eba-9a7f-8a8cd0c740d0 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:09.740256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:09.846348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e1ea4-ab6d-4eba-9a7f-8a8cd0c740d0 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:09.901062 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563eb6f-5c71-4a2a-ae2f-b93dab170a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:09.902011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563eb6f-5c71-4a2a-ae2f-b93dab170a11 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:09.902298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563eb6f-5c71-4a2a-ae2f-b93dab170a11 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:09.902616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563eb6f-5c71-4a2a-ae2f-b93dab170a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:09.919683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563eb6f-5c71-4a2a-ae2f-b93dab170a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:09.921164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563eb6f-5c71-4a2a-ae2f-b93dab170a11 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:09.937148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7addac4-3956-4083-b30a-2eb4e7a7ad31 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:09.937716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7addac4-3956-4083-b30a-2eb4e7a7ad31 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:09.937716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7addac4-3956-4083-b30a-2eb4e7a7ad31 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:09.939110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 258/485] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-8563eb6f-5c71-4a2a-ae2f-b93dab170a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:09.952250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:09.953799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:09.954654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:09.954654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:09.954654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:09.954654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:09.954989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf043f8-6b49-4eea-b4eb-3ce3419fd381 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:09.954989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf043f8-6b49-4eea-b4eb-3ce3419fd381 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:09.954989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf043f8-6b49-4eea-b4eb-3ce3419fd381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:09.955282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 228/486] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:09.963572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563eb6f-5c71-4a2a-ae2f-b93dab170a11 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:10.002502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-5cf043f8-6b49-4eea-b4eb-3ce3419fd381 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:10.003728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5cf043f8-6b49-4eea-b4eb-3ce3419fd381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:10.091124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563eb6f-5c71-4a2a-ae2f-b93dab170a11 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:10.117812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 259/487] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:35:10.119421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76487bfb-0a6e-44ef-9bd8-3fab141ebcb4 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:10.119594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76487bfb-0a6e-44ef-9bd8-3fab141ebcb4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:10.119895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76487bfb-0a6e-44ef-9bd8-3fab141ebcb4 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:10.122136 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 260/488] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:10.124063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:10.139078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d7860-f953-4f68-8d43-9b92e70d84bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:10.140035 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfef9d-137b-459b-b268-e8be6b3185dc None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:10.140327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfef9d-137b-459b-b268-e8be6b3185dc None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:10.140569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d7860-f953-4f68-8d43-9b92e70d84bc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:10.140691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfef9d-137b-459b-b268-e8be6b3185dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:10.141197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d7860-f953-4f68-8d43-9b92e70d84bc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:10.141726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d7860-f953-4f68-8d43-9b92e70d84bc None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:10.142672 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d7860-f953-4f68-8d43-9b92e70d84bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:10.143234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d7860-f953-4f68-8d43-9b92e70d84bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:10.144871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d7860-f953-4f68-8d43-9b92e70d84bc None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:10.161041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 229/489] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:10.162352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d7860-f953-4f68-8d43-9b92e70d84bc None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:10.168821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-1ebfef9d-137b-459b-b268-e8be6b3185dc None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:10.170059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1ebfef9d-137b-459b-b268-e8be6b3185dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:10.229847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:10.257444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 261/490] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:35:10.258998 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfef9d-137b-459b-b268-e8be6b3185dc None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:10.305673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d7860-f953-4f68-8d43-9b92e70d84bc None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:10.347976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e61b3-f6d9-41a8-8581-1551007aabcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:10.348847 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e61b3-f6d9-41a8-8581-1551007aabcf None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:10.349076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e61b3-f6d9-41a8-8581-1551007aabcf None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:10.349467 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e61b3-f6d9-41a8-8581-1551007aabcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:10.363197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e61b3-f6d9-41a8-8581-1551007aabcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-6a0e61b3-f6d9-41a8-8581-1551007aabcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:10.373446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:10.374574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:10.375623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 230/491] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:10.376276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e61b3-f6d9-41a8-8581-1551007aabcf None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:10.407277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:10.422629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfef9d-137b-459b-b268-e8be6b3185dc None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:10.480008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdcb1f-0925-4e82-8c8a-1308f9e888fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:10.482156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdcb1f-0925-4e82-8c8a-1308f9e888fd None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:10.482458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdcb1f-0925-4e82-8c8a-1308f9e888fd None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:10.482742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdcb1f-0925-4e82-8c8a-1308f9e888fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:10.493753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fdcb1f-0925-4e82-8c8a-1308f9e888fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:10.495274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fdcb1f-0925-4e82-8c8a-1308f9e888fd None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-f1fdcb1f-0925-4e82-8c8a-1308f9e888fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:10.517037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:10.518356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:10.519317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:10.519317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:10.519317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:10.519317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:10.519317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:10.519317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:10.519317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:10.519317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:10.519317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:10.519317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:10.519317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 262/492] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jul 19 20:35:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:10.519956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdcb1f-0925-4e82-8c8a-1308f9e888fd None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:10.537545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e61b3-f6d9-41a8-8581-1551007aabcf None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:10.582923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd415f2-2cc9-4759-8efa-24e5204812a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:10.584286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd415f2-2cc9-4759-8efa-24e5204812a3 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:10.584810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd415f2-2cc9-4759-8efa-24e5204812a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:10.585302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd415f2-2cc9-4759-8efa-24e5204812a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:10.585813 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:10.609148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd415f2-2cc9-4759-8efa-24e5204812a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-4bd415f2-2cc9-4759-8efa-24e5204812a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:10.623900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:10.624929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:10.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:10.628481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 231/493] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:10.629552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd415f2-2cc9-4759-8efa-24e5204812a3 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:10.672082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdcb1f-0925-4e82-8c8a-1308f9e888fd None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:10.717171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a52da-e0cd-41e1-88c1-116c87715607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:10.717994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a52da-e0cd-41e1-88c1-116c87715607 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:10.718280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a52da-e0cd-41e1-88c1-116c87715607 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:10.718567 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a52da-e0cd-41e1-88c1-116c87715607 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:10.719624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a52da-e0cd-41e1-88c1-116c87715607 None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:10.719994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a52da-e0cd-41e1-88c1-116c87715607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:10.721660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a52da-e0cd-41e1-88c1-116c87715607 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:10.724957 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:10.734483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 263/494] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:10.735940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5f19-aaaa-4414-ab11-63ac36630dbd None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:10.736230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5f19-aaaa-4414-ab11-63ac36630dbd None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:10.736510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da5f19-aaaa-4414-ab11-63ac36630dbd None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:10.738267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 264/495] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:10.738612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da5f19-aaaa-4414-ab11-63ac36630dbd None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:10.767688 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd415f2-2cc9-4759-8efa-24e5204812a3 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:10.819924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa00c1-e8ba-4f63-98f9-d36f65eadb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:10.821262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa00c1-e8ba-4f63-98f9-d36f65eadb36 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:10.821848 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa00c1-e8ba-4f63-98f9-d36f65eadb36 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:10.822403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa00c1-e8ba-4f63-98f9-d36f65eadb36 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:10.824241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa00c1-e8ba-4f63-98f9-d36f65eadb36 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:10.825470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa00c1-e8ba-4f63-98f9-d36f65eadb36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:10.828298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa00c1-e8ba-4f63-98f9-d36f65eadb36 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:10.893635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da5f19-aaaa-4414-ab11-63ac36630dbd None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:10.900564 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 232/496] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jul 19 20:35:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:10.900564 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa00c1-e8ba-4f63-98f9-d36f65eadb36 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:10.910196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:10.952602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddc547-d4d1-4cb0-92a9-e16803c32412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:10.953800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddc547-d4d1-4cb0-92a9-e16803c32412 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:10.954145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddc547-d4d1-4cb0-92a9-e16803c32412 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:10.954554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddc547-d4d1-4cb0-92a9-e16803c32412 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:10.955832 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddc547-d4d1-4cb0-92a9-e16803c32412 None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:10.956371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddc547-d4d1-4cb0-92a9-e16803c32412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:10.958823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddc547-d4d1-4cb0-92a9-e16803c32412 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:10.972174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 265/497] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:10.974272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a0e28-b738-45f6-a24d-d529c64d5632 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:10.974745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a0e28-b738-45f6-a24d-d529c64d5632 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:10.975165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a0e28-b738-45f6-a24d-d529c64d5632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:10.999384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-161a0e28-b738-45f6-a24d-d529c64d5632 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:11.001074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-161a0e28-b738-45f6-a24d-d529c64d5632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:11.069888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa00c1-e8ba-4f63-98f9-d36f65eadb36 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:11.095754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 266/498] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:11.097588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a0e28-b738-45f6-a24d-d529c64d5632 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:11.115978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:11.122768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746fa53-de3f-4535-b09a-72f5d62a0526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:11.123696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746fa53-de3f-4535-b09a-72f5d62a0526 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:11.123983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746fa53-de3f-4535-b09a-72f5d62a0526 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:11.124263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746fa53-de3f-4535-b09a-72f5d62a0526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:11.142736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746fa53-de3f-4535-b09a-72f5d62a0526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:11.144558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746fa53-de3f-4535-b09a-72f5d62a0526 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-d746fa53-de3f-4535-b09a-72f5d62a0526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:11.164117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:11.165338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:11.171917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:11.171917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:11.171917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:11.171917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:11.171917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 233/499] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:11.171917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746fa53-de3f-4535-b09a-72f5d62a0526 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:11.245107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a0e28-b738-45f6-a24d-d529c64d5632 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:11.247223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:11.288439 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746fa53-de3f-4535-b09a-72f5d62a0526 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:11.290637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72103de3-c580-4413-b562-0e1912fdbbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:11.291579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72103de3-c580-4413-b562-0e1912fdbbbb None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:11.292033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72103de3-c580-4413-b562-0e1912fdbbbb None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:11.292429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72103de3-c580-4413-b562-0e1912fdbbbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:11.338668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247399cf-1463-4666-9250-33175655b84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:11.339480 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247399cf-1463-4666-9250-33175655b84f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:11.339803 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247399cf-1463-4666-9250-33175655b84f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:11.340100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247399cf-1463-4666-9250-33175655b84f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:11.352462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247399cf-1463-4666-9250-33175655b84f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:11.354235 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247399cf-1463-4666-9250-33175655b84f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-247399cf-1463-4666-9250-33175655b84f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:11.376231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:11.377973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:11.379453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:11.379453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:11.379453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:11.379453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:11.379453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 234/500] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:11.379453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93f14b-c0b1-4291-8861-dcc9b8e06f0a None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:11.379984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93f14b-c0b1-4291-8861-dcc9b8e06f0a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:11.380085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93f14b-c0b1-4291-8861-dcc9b8e06f0a None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:11.381775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 235/501] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:11.382935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93f14b-c0b1-4291-8861-dcc9b8e06f0a None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:11.399457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72103de3-c580-4413-b562-0e1912fdbbbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:11.401042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72103de3-c580-4413-b562-0e1912fdbbbb None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:11.433841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:11.525829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93f14b-c0b1-4291-8861-dcc9b8e06f0a None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:11.537827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 267/502] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:11.538775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72103de3-c580-4413-b562-0e1912fdbbbb None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:11.541338 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:11.582601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb452c7-eb3d-4f39-a5ca-f15e9ae16961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:11.583516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb452c7-eb3d-4f39-a5ca-f15e9ae16961 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:11.583803 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb452c7-eb3d-4f39-a5ca-f15e9ae16961 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:11.584164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb452c7-eb3d-4f39-a5ca-f15e9ae16961 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:11.598143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb452c7-eb3d-4f39-a5ca-f15e9ae16961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-2eb452c7-eb3d-4f39-a5ca-f15e9ae16961 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:11.607918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:11.617852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 236/503] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:11.619490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb452c7-eb3d-4f39-a5ca-f15e9ae16961 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:11.661812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72103de3-c580-4413-b562-0e1912fdbbbb None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:11.709144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f7440-70a6-40c4-9e38-959360817709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:11.710017 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f7440-70a6-40c4-9e38-959360817709 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:11.710389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f7440-70a6-40c4-9e38-959360817709 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:11.710755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f7440-70a6-40c4-9e38-959360817709 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:11.711636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f7440-70a6-40c4-9e38-959360817709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:11.712034 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f7440-70a6-40c4-9e38-959360817709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:11.713616 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f7440-70a6-40c4-9e38-959360817709 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:11.728863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 268/504] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:11.731407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237c647-6381-45c7-b28a-c2f9644767f0 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:11.731884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237c647-6381-45c7-b28a-c2f9644767f0 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:11.732341 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237c647-6381-45c7-b28a-c2f9644767f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:11.739534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:11.747610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb452c7-eb3d-4f39-a5ca-f15e9ae16961 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:11.752655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-1237c647-6381-45c7-b28a-c2f9644767f0 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:11.753943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1237c647-6381-45c7-b28a-c2f9644767f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:11.789793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43476581-2c6e-42cb-a3bd-3973b52bfe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:11.790594 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43476581-2c6e-42cb-a3bd-3973b52bfe5d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:11.791103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43476581-2c6e-42cb-a3bd-3973b52bfe5d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:11.791530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43476581-2c6e-42cb-a3bd-3973b52bfe5d None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:11.792517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43476581-2c6e-42cb-a3bd-3973b52bfe5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:11.793031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43476581-2c6e-42cb-a3bd-3973b52bfe5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:11.794638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43476581-2c6e-42cb-a3bd-3973b52bfe5d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:11.809018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 237/505] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:11.810154 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43476581-2c6e-42cb-a3bd-3973b52bfe5d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:11.828539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:11.841943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 269/506] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:11.842806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237c647-6381-45c7-b28a-c2f9644767f0 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:11.951458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43476581-2c6e-42cb-a3bd-3973b52bfe5d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:11.970363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237c647-6381-45c7-b28a-c2f9644767f0 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:11.993559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be1d30-d1fd-49cd-8f87-5084d065a769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:11.994532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be1d30-d1fd-49cd-8f87-5084d065a769 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:11.994880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be1d30-d1fd-49cd-8f87-5084d065a769 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:11.995215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be1d30-d1fd-49cd-8f87-5084d065a769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:12.008097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be1d30-d1fd-49cd-8f87-5084d065a769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:12.010076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be1d30-d1fd-49cd-8f87-5084d065a769 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:12.014565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488209e4-4e54-483f-8f7a-06fa6e33307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:12.015259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488209e4-4e54-483f-8f7a-06fa6e33307e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:12.015539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488209e4-4e54-483f-8f7a-06fa6e33307e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:12.015814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488209e4-4e54-483f-8f7a-06fa6e33307e None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:12.016580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488209e4-4e54-483f-8f7a-06fa6e33307e None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:12.016889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488209e4-4e54-483f-8f7a-06fa6e33307e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:12.018601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488209e4-4e54-483f-8f7a-06fa6e33307e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:12.028264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 270/507] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:12.029152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488209e4-4e54-483f-8f7a-06fa6e33307e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-18be1d30-d1fd-49cd-8f87-5084d065a769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:12.040798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:12.042728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:12.044388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:12.044388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:12.044388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:12.044388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:12.044388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 238/508] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:12.044388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be1d30-d1fd-49cd-8f87-5084d065a769 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:12.066206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:12.117659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:12.185907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488209e4-4e54-483f-8f7a-06fa6e33307e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:12.186181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be1d30-d1fd-49cd-8f87-5084d065a769 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:12.224426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b93f55-6ca8-426d-be4b-917c4ec9e99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:12.225149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b93f55-6ca8-426d-be4b-917c4ec9e99b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:12.225468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b93f55-6ca8-426d-be4b-917c4ec9e99b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:12.225749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b93f55-6ca8-426d-be4b-917c4ec9e99b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:12.227919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05069d-9b8c-4980-a74b-6bd81fa422bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:12.228748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05069d-9b8c-4980-a74b-6bd81fa422bf None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:12.229037 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05069d-9b8c-4980-a74b-6bd81fa422bf None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:12.229325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05069d-9b8c-4980-a74b-6bd81fa422bf None admin] PATH_INFO: `/v3/users/glance` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:12.235129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b93f55-6ca8-426d-be4b-917c4ec9e99b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:12.236597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b93f55-6ca8-426d-be4b-917c4ec9e99b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:12.237764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05069d-9b8c-4980-a74b-6bd81fa422bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:12.239419 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05069d-9b8c-4980-a74b-6bd81fa422bf None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-91b93f55-6ca8-426d-be4b-917c4ec9e99b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:12.252828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:12.254672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 271/509] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:12.256415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b93f55-6ca8-426d-be4b-917c4ec9e99b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-2d05069d-9b8c-4980-a74b-6bd81fa422bf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:12.257664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:12.259241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 239/510] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:12.260999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05069d-9b8c-4980-a74b-6bd81fa422bf None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:12.347370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:12.391782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b93f55-6ca8-426d-be4b-917c4ec9e99b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:12.392213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05069d-9b8c-4980-a74b-6bd81fa422bf None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:12.407685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:12.430090 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c94d7d-b141-494c-b77a-0c909433890b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:12.431003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c94d7d-b141-494c-b77a-0c909433890b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:12.431286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c94d7d-b141-494c-b77a-0c909433890b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:12.431561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c94d7d-b141-494c-b77a-0c909433890b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:12.445582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aee817-4409-442b-b120-c1e673ff73fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:12.446252 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c94d7d-b141-494c-b77a-0c909433890b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:12.446904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aee817-4409-442b-b120-c1e673ff73fc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:12.448275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aee817-4409-442b-b120-c1e673ff73fc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:12.448275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aee817-4409-442b-b120-c1e673ff73fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:12.449197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c94d7d-b141-494c-b77a-0c909433890b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:12.473429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aee817-4409-442b-b120-c1e673ff73fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:12.476111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aee817-4409-442b-b120-c1e673ff73fc None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-36c94d7d-b141-494c-b77a-0c909433890b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:12.478515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:12.480911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:12.481931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:12.481931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:12.481931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:12.481931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:12.481931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 272/511] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:12.482551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c94d7d-b141-494c-b77a-0c909433890b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-b3aee817-4409-442b-b120-c1e673ff73fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:12.514539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:12.516283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:12.517904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:12.517904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:12.517904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:12.517904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:12.517904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 240/512] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:12.518495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aee817-4409-442b-b120-c1e673ff73fc None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:12.646200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c94d7d-b141-494c-b77a-0c909433890b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:12.666452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aee817-4409-442b-b120-c1e673ff73fc None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:12.689970 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5fd1a-71cd-4117-a1e9-fb91c44aeb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:12.691029 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5fd1a-71cd-4117-a1e9-fb91c44aeb17 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:12.691492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5fd1a-71cd-4117-a1e9-fb91c44aeb17 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:12.691950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5fd1a-71cd-4117-a1e9-fb91c44aeb17 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:12.693193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5fd1a-71cd-4117-a1e9-fb91c44aeb17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:12.693769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5fd1a-71cd-4117-a1e9-fb91c44aeb17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:12.696026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5fd1a-71cd-4117-a1e9-fb91c44aeb17 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:12.699983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:12.711781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 273/513] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:12.712516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5fd1a-71cd-4117-a1e9-fb91c44aeb17 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:12.722664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5469295-43e3-4651-963b-f2cba14bdb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:12.723451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5469295-43e3-4651-963b-f2cba14bdb87 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:12.723825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5469295-43e3-4651-963b-f2cba14bdb87 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:12.724287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5469295-43e3-4651-963b-f2cba14bdb87 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:12.725283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5469295-43e3-4651-963b-f2cba14bdb87 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:12.725877 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5469295-43e3-4651-963b-f2cba14bdb87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:12.727932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5469295-43e3-4651-963b-f2cba14bdb87 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:12.756406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:12.759479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 241/514] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jul 19 20:35:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:12.760874 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5469295-43e3-4651-963b-f2cba14bdb87 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:12.852465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5fd1a-71cd-4117-a1e9-fb91c44aeb17 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:12.896407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5469295-43e3-4651-963b-f2cba14bdb87 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:12.898317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c425f-8e3f-4d86-9411-398f99d872f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:12.899475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c425f-8e3f-4d86-9411-398f99d872f5 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:12.899933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c425f-8e3f-4d86-9411-398f99d872f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:12.900394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c425f-8e3f-4d86-9411-398f99d872f5 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:12.902093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c425f-8e3f-4d86-9411-398f99d872f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:12.903102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c425f-8e3f-4d86-9411-398f99d872f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:12.905955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c425f-8e3f-4d86-9411-398f99d872f5 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:12.937278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fac32d-c045-4551-bdba-e4d7977fd3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:12.938119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac32d-c045-4551-bdba-e4d7977fd3d5 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:12.938410 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac32d-c045-4551-bdba-e4d7977fd3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:12.938686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac32d-c045-4551-bdba-e4d7977fd3d5 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:12.939421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fac32d-c045-4551-bdba-e4d7977fd3d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:12.939737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fac32d-c045-4551-bdba-e4d7977fd3d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:12.941112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fac32d-c045-4551-bdba-e4d7977fd3d5 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:12.953861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 242/515] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:12.954706 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fac32d-c045-4551-bdba-e4d7977fd3d5 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:12.968748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 274/516] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jul 19 20:35:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:12.969958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c425f-8e3f-4d86-9411-398f99d872f5 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:13.016889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:13.077868 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:13.101166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fac32d-c045-4551-bdba-e4d7977fd3d5 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:13.102133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c425f-8e3f-4d86-9411-398f99d872f5 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:13.158451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec91d5-bf6f-4940-a4ee-39df65a9d007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:13.159556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec91d5-bf6f-4940-a4ee-39df65a9d007 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:13.160034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec91d5-bf6f-4940-a4ee-39df65a9d007 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:13.160509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec91d5-bf6f-4940-a4ee-39df65a9d007 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:13.161916 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec91d5-bf6f-4940-a4ee-39df65a9d007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:13.163014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec91d5-bf6f-4940-a4ee-39df65a9d007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:13.163208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5aa0d-41a3-42b1-adcc-059b11e85876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:13.163582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5aa0d-41a3-42b1-adcc-059b11e85876 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:13.163877 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5aa0d-41a3-42b1-adcc-059b11e85876 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:13.164159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5aa0d-41a3-42b1-adcc-059b11e85876 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:13.165014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec91d5-bf6f-4940-a4ee-39df65a9d007 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:13.172350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5aa0d-41a3-42b1-adcc-059b11e85876 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:13.173904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5aa0d-41a3-42b1-adcc-059b11e85876 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:13.194254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 243/517] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-0ce5aa0d-41a3-42b1-adcc-059b11e85876 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:13.195048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:13.196420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:13.197858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:13.197858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:13.197858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:13.197858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:13.197858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:13.197858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:13.197858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:13.197858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:13.198486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec91d5-bf6f-4940-a4ee-39df65a9d007 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:13.198582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 275/518] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:13.198582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5aa0d-41a3-42b1-adcc-059b11e85876 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:13.314336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:13.320032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec91d5-bf6f-4940-a4ee-39df65a9d007 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:13.338977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5aa0d-41a3-42b1-adcc-059b11e85876 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:13.367142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb860e6-dc6e-4554-86d5-98a39855c912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:13.367969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb860e6-dc6e-4554-86d5-98a39855c912 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:13.368137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb860e6-dc6e-4554-86d5-98a39855c912 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:13.368395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb860e6-dc6e-4554-86d5-98a39855c912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:13.387095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10baeb-2411-4b73-a7f8-89020f941772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:13.388132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10baeb-2411-4b73-a7f8-89020f941772 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:13.388610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10baeb-2411-4b73-a7f8-89020f941772 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:13.389053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10baeb-2411-4b73-a7f8-89020f941772 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:13.390225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10baeb-2411-4b73-a7f8-89020f941772 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:13.392942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10baeb-2411-4b73-a7f8-89020f941772 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:13.415933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:13.423229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 276/519] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:35:13 2022] POST /identity/v3/users => generated 247 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:13.425116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10baeb-2411-4b73-a7f8-89020f941772 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:13.489431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb860e6-dc6e-4554-86d5-98a39855c912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:13.493390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb860e6-dc6e-4554-86d5-98a39855c912 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:13.601478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10baeb-2411-4b73-a7f8-89020f941772 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:13.645168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:13.658390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 244/520] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:13.658623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b3bb0-706b-423d-a05e-1bfdd7ee74c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:13.658623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b3bb0-706b-423d-a05e-1bfdd7ee74c3 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:13.658907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb860e6-dc6e-4554-86d5-98a39855c912 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:13.659019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b3bb0-706b-423d-a05e-1bfdd7ee74c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:13.659019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b3bb0-706b-423d-a05e-1bfdd7ee74c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:13.774014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:13.780070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b3bb0-706b-423d-a05e-1bfdd7ee74c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:13.782824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b3bb0-706b-423d-a05e-1bfdd7ee74c3 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:13.847781 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb860e6-dc6e-4554-86d5-98a39855c912 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:13.910942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb234a7-e67f-44c8-a2c4-72c5e71440f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:13.912024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb234a7-e67f-44c8-a2c4-72c5e71440f3 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:13.912488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb234a7-e67f-44c8-a2c4-72c5e71440f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:13.912880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb234a7-e67f-44c8-a2c4-72c5e71440f3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:13.929964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb234a7-e67f-44c8-a2c4-72c5e71440f3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:13.931453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb234a7-e67f-44c8-a2c4-72c5e71440f3 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-9fb234a7-e67f-44c8-a2c4-72c5e71440f3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:13.950192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:13.951379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:13.952715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:13.952715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:13.952715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:13.952715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:13.952715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:13.952715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:13.952715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:13.952715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:13.952715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:35:13.952715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:13.956236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 245/521] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:13.957207 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb234a7-e67f-44c8-a2c4-72c5e71440f3 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:13.981538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 277/522] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:13.983301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b3bb0-706b-423d-a05e-1bfdd7ee74c3 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:14.058673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:14.128615 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb234a7-e67f-44c8-a2c4-72c5e71440f3 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:14.147978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:14.156056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b3bb0-706b-423d-a05e-1bfdd7ee74c3 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:14.202979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20474f22-5552-4430-9523-c29043e78ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:14.203978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20474f22-5552-4430-9523-c29043e78ee5 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:14.204429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20474f22-5552-4430-9523-c29043e78ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:14.204871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20474f22-5552-4430-9523-c29043e78ee5 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:14.206283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20474f22-5552-4430-9523-c29043e78ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:14.206925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20474f22-5552-4430-9523-c29043e78ee5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:14.209429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20474f22-5552-4430-9523-c29043e78ee5 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:14.212074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e2aad-0695-4105-88cd-e885cce130ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:14.213384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e2aad-0695-4105-88cd-e885cce130ec None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:14.213917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e2aad-0695-4105-88cd-e885cce130ec None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:14.214399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e2aad-0695-4105-88cd-e885cce130ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:14.233792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e2aad-0695-4105-88cd-e885cce130ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:14.236645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e2aad-0695-4105-88cd-e885cce130ec None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:14.248716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 246/523] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:14.248716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20474f22-5552-4430-9523-c29043e78ee5 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-5c0e2aad-0695-4105-88cd-e885cce130ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:14.271070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:14.271993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:14.272814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:14.272814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:14.272814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:14.272814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:14.273996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 278/524] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:14.275029 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e2aad-0695-4105-88cd-e885cce130ec None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:14.435258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e2aad-0695-4105-88cd-e885cce130ec None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:14.448122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20474f22-5552-4430-9523-c29043e78ee5 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:14.482580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:14.491881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be5277-1e2b-45ef-8be6-20dd8985a5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:14.492869 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be5277-1e2b-45ef-8be6-20dd8985a5ed None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:14.493406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be5277-1e2b-45ef-8be6-20dd8985a5ed None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:14.493956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be5277-1e2b-45ef-8be6-20dd8985a5ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:14.510919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be5277-1e2b-45ef-8be6-20dd8985a5ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:14.513553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be5277-1e2b-45ef-8be6-20dd8985a5ed None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:14.514012 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a23d80-2003-419d-a544-b955a5582367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:14.515063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a23d80-2003-419d-a544-b955a5582367 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:14.515549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a23d80-2003-419d-a544-b955a5582367 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:14.516030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a23d80-2003-419d-a544-b955a5582367 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:14.517936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a23d80-2003-419d-a544-b955a5582367 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:14.518567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a23d80-2003-419d-a544-b955a5582367 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:14.521176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a23d80-2003-419d-a544-b955a5582367 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-d3be5277-1e2b-45ef-8be6-20dd8985a5ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:14.540269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:14.542013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:14.551464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:14.551464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:14.551464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:14.551464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:14.551464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 279/525] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:14.551464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be5277-1e2b-45ef-8be6-20dd8985a5ed None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:14.554145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 247/526] 10.222.0.70 () {56 vars in 1139 bytes} [Tue Jul 19 20:35:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:14.561768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a23d80-2003-419d-a544-b955a5582367 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:14.562166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:14.708283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be5277-1e2b-45ef-8be6-20dd8985a5ed None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:14.721852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a23d80-2003-419d-a544-b955a5582367 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:14.769799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6e09c-34b5-45e9-bb17-aaa8df92f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:14.770660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6e09c-34b5-45e9-bb17-aaa8df92f8ab None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:14.771073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6e09c-34b5-45e9-bb17-aaa8df92f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:14.771518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6e09c-34b5-45e9-bb17-aaa8df92f8ab None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:14.772466 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6e09c-34b5-45e9-bb17-aaa8df92f8ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:14.772955 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6e09c-34b5-45e9-bb17-aaa8df92f8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:14.774642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6e09c-34b5-45e9-bb17-aaa8df92f8ab None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:14.776806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef558b6-8a85-42fe-a97a-9e32850e0fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:14.778711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef558b6-8a85-42fe-a97a-9e32850e0fda None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:14.779002 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef558b6-8a85-42fe-a97a-9e32850e0fda None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:14.779475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef558b6-8a85-42fe-a97a-9e32850e0fda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:14.780978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef558b6-8a85-42fe-a97a-9e32850e0fda None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=e1d09e168fc54f1da06b719989ae8ef0) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:14.781569 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef558b6-8a85-42fe-a97a-9e32850e0fda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:14.784322 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef558b6-8a85-42fe-a97a-9e32850e0fda None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:14.789961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 248/527] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:14.805623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 280/528] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:14 2022] GET /identity/v3/role_assignments?user.id=e1d09e168fc54f1da06b719989ae8ef0&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:14.805893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6e09c-34b5-45e9-bb17-aaa8df92f8ab None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:14.806044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef558b6-8a85-42fe-a97a-9e32850e0fda None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:14.870539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:14.937721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:14.993952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6e09c-34b5-45e9-bb17-aaa8df92f8ab None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:15.008208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef558b6-8a85-42fe-a97a-9e32850e0fda None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:15.068362 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3a2c0-60ff-44c3-9ae0-f3c1aa935a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:15.069449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3a2c0-60ff-44c3-9ae0-f3c1aa935a14 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:15.069909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3a2c0-60ff-44c3-9ae0-f3c1aa935a14 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:15.070358 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3a2c0-60ff-44c3-9ae0-f3c1aa935a14 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:15.072937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3a2c0-60ff-44c3-9ae0-f3c1aa935a14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:15.072937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3a2c0-60ff-44c3-9ae0-f3c1aa935a14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:15.076046 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3a2c0-60ff-44c3-9ae0-f3c1aa935a14 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:15.081807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fac2f-b0bb-448a-976e-8541ec06213e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:15.083022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fac2f-b0bb-448a-976e-8541ec06213e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:15.083579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fac2f-b0bb-448a-976e-8541ec06213e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:15.084056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fac2f-b0bb-448a-976e-8541ec06213e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:15.096052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 249/529] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:15.097548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3a2c0-60ff-44c3-9ae0-f3c1aa935a14 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:15.233915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fac2f-b0bb-448a-976e-8541ec06213e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:15.236697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fac2f-b0bb-448a-976e-8541ec06213e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:15.274812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:15.288606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3a2c0-60ff-44c3-9ae0-f3c1aa935a14 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:15.351661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:15.358509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301275b-c82e-4044-88b0-10a9f67d39da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:15.359870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301275b-c82e-4044-88b0-10a9f67d39da None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:15.360370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301275b-c82e-4044-88b0-10a9f67d39da None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:15.360870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301275b-c82e-4044-88b0-10a9f67d39da None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:15.387021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301275b-c82e-4044-88b0-10a9f67d39da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:15.390216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301275b-c82e-4044-88b0-10a9f67d39da None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-6301275b-c82e-4044-88b0-10a9f67d39da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:15.420462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:15.422376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:15.424264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:15.424264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:15.424264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:15.424264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:15.424264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:15.424264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:15.424264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:15.424264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:15.428071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 250/530] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:15.429875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301275b-c82e-4044-88b0-10a9f67d39da None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:15.430650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 281/531] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:15.434691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fac2f-b0bb-448a-976e-8541ec06213e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:15.604616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301275b-c82e-4044-88b0-10a9f67d39da None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:15.627728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fac2f-b0bb-448a-976e-8541ec06213e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:15.666662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e06bdc-c33c-4ff1-9cde-76c4bd0ad093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:15.669021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e06bdc-c33c-4ff1-9cde-76c4bd0ad093 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:15.669505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e06bdc-c33c-4ff1-9cde-76c4bd0ad093 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:15.669941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e06bdc-c33c-4ff1-9cde-76c4bd0ad093 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:15.682311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e06bdc-c33c-4ff1-9cde-76c4bd0ad093 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:15.686254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e06bdc-c33c-4ff1-9cde-76c4bd0ad093 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:15.699381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0ce44-459e-437d-8d03-7056aa34c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:15.701012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0ce44-459e-437d-8d03-7056aa34c647 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:15.701837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0ce44-459e-437d-8d03-7056aa34c647 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:15.702614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0ce44-459e-437d-8d03-7056aa34c647 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:15.704757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0ce44-459e-437d-8d03-7056aa34c647 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:15.705802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0ce44-459e-437d-8d03-7056aa34c647 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:15.708883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0ce44-459e-437d-8d03-7056aa34c647 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-03e06bdc-c33c-4ff1-9cde-76c4bd0ad093 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:15.711268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:15.712860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:15.714656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:15.714656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:15.714656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:15.714656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:15.714656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:15.714656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:15.714656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:15.714656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:15.720756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 251/532] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:15.722218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e06bdc-c33c-4ff1-9cde-76c4bd0ad093 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:15.756975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 282/533] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:15.758910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0ce44-459e-437d-8d03-7056aa34c647 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:15.764611 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:15.819203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:15.923628 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e06bdc-c33c-4ff1-9cde-76c4bd0ad093 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:15.974959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0ce44-459e-437d-8d03-7056aa34c647 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:15.986974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2375fbd-3589-4d9a-84b6-b2e751fb8f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:15.988373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2375fbd-3589-4d9a-84b6-b2e751fb8f87 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:15.988869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2375fbd-3589-4d9a-84b6-b2e751fb8f87 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:15.989383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2375fbd-3589-4d9a-84b6-b2e751fb8f87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:16.011637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2375fbd-3589-4d9a-84b6-b2e751fb8f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:16.014399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2375fbd-3589-4d9a-84b6-b2e751fb8f87 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:16.045147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6aa47-ddf4-45e5-879b-7546c3aa0468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:16.046863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6aa47-ddf4-45e5-879b-7546c3aa0468 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:16.047653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6aa47-ddf4-45e5-879b-7546c3aa0468 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:16.048416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6aa47-ddf4-45e5-879b-7546c3aa0468 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:16.050245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6aa47-ddf4-45e5-879b-7546c3aa0468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:16.051075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6aa47-ddf4-45e5-879b-7546c3aa0468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:16.054088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6aa47-ddf4-45e5-879b-7546c3aa0468 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-f2375fbd-3589-4d9a-84b6-b2e751fb8f87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:16.057731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:16.059411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:16.061103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:16.061103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:16.061103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:16.061103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:16.065575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 252/534] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:16.067445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2375fbd-3589-4d9a-84b6-b2e751fb8f87 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:16.086619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 283/535] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:16.088529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6aa47-ddf4-45e5-879b-7546c3aa0468 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:16.179161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:16.223343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2375fbd-3589-4d9a-84b6-b2e751fb8f87 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:16.228282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:16.246187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6aa47-ddf4-45e5-879b-7546c3aa0468 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:16.289758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b81c61-8672-4908-9139-cfb6bb5e292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:16.291754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b81c61-8672-4908-9139-cfb6bb5e292b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:16.292191 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b81c61-8672-4908-9139-cfb6bb5e292b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:16.292623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b81c61-8672-4908-9139-cfb6bb5e292b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:16.294023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b81c61-8672-4908-9139-cfb6bb5e292b None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=a75951e9512c420992a4fd6c1a6b1176) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:16.294522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b81c61-8672-4908-9139-cfb6bb5e292b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:16.297044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b81c61-8672-4908-9139-cfb6bb5e292b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:16.299727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126b610-e12d-4665-a1ae-6873bb3a7a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:16.300542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126b610-e12d-4665-a1ae-6873bb3a7a08 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:16.300722 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126b610-e12d-4665-a1ae-6873bb3a7a08 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:16.300971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126b610-e12d-4665-a1ae-6873bb3a7a08 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:16.301792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126b610-e12d-4665-a1ae-6873bb3a7a08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:16.302115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126b610-e12d-4665-a1ae-6873bb3a7a08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:16.303948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126b610-e12d-4665-a1ae-6873bb3a7a08 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:16.317409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 253/536] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:16 2022] GET /identity/v3/role_assignments?user.id=a75951e9512c420992a4fd6c1a6b1176&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:16.317409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b81c61-8672-4908-9139-cfb6bb5e292b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:16.330945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 284/537] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:16.333532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78de1b-1545-45ae-83b9-918ddc6508ea None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:16.334083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78de1b-1545-45ae-83b9-918ddc6508ea None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:16.334605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78de1b-1545-45ae-83b9-918ddc6508ea None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:16.344451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 285/538] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:16 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:16.346289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78de1b-1545-45ae-83b9-918ddc6508ea None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:16.495770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b81c61-8672-4908-9139-cfb6bb5e292b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:16.533961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:16.540651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a4db7-e204-4ce4-947e-0dad42c46902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:16.542012 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a4db7-e204-4ce4-947e-0dad42c46902 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:16.542517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a4db7-e204-4ce4-947e-0dad42c46902 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:16.543043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a4db7-e204-4ce4-947e-0dad42c46902 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:16.544292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a4db7-e204-4ce4-947e-0dad42c46902 None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=82e1192b2d554199a08debb4aa392479) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:16.544829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a4db7-e204-4ce4-947e-0dad42c46902 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:16.547008 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a4db7-e204-4ce4-947e-0dad42c46902 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:16.554787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 254/539] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:16 2022] GET /identity/v3/role_assignments?user.id=82e1192b2d554199a08debb4aa392479&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:16.556500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a359f-9981-4686-80b3-6cdb45eee74a None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:16.557687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a359f-9981-4686-80b3-6cdb45eee74a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:16.558216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a359f-9981-4686-80b3-6cdb45eee74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:16.580325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-911a359f-9981-4686-80b3-6cdb45eee74a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:16.580528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78de1b-1545-45ae-83b9-918ddc6508ea None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:16.582177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-911a359f-9981-4686-80b3-6cdb45eee74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:16.662654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc70d2b-f1aa-4aec-a8d1-77a936a12fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:16.664614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc70d2b-f1aa-4aec-a8d1-77a936a12fca None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:16.666387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc70d2b-f1aa-4aec-a8d1-77a936a12fca None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:16.666909 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc70d2b-f1aa-4aec-a8d1-77a936a12fca None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:16.676076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 255/540] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:16.691134 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc70d2b-f1aa-4aec-a8d1-77a936a12fca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:16.692823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc70d2b-f1aa-4aec-a8d1-77a936a12fca None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-ddc70d2b-f1aa-4aec-a8d1-77a936a12fca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:16.710160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:16.712175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:16.714481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:16.714481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:16.714481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:16.714481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:16.714481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:16.714481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:16.714481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:16.714481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:16.714481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 286/541] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:16.720760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a359f-9981-4686-80b3-6cdb45eee74a None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:16.723596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc70d2b-f1aa-4aec-a8d1-77a936a12fca None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:16.739465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:16.881398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc70d2b-f1aa-4aec-a8d1-77a936a12fca None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:16.934044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a359f-9981-4686-80b3-6cdb45eee74a None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:16.966237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93f07c-c654-429d-81c7-48f359afd2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:16.966940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93f07c-c654-429d-81c7-48f359afd2c9 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:16.967225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93f07c-c654-429d-81c7-48f359afd2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:16.967502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93f07c-c654-429d-81c7-48f359afd2c9 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:16.968320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93f07c-c654-429d-81c7-48f359afd2c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:16.968634 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93f07c-c654-429d-81c7-48f359afd2c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:16.970390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93f07c-c654-429d-81c7-48f359afd2c9 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:16.977743 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:16.990783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b30ee4-fe4c-4d54-8b20-862a5b238eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:16.991739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b30ee4-fe4c-4d54-8b20-862a5b238eae None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:16.992077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b30ee4-fe4c-4d54-8b20-862a5b238eae None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:16.992407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b30ee4-fe4c-4d54-8b20-862a5b238eae None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:17.006982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 287/542] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:17.013124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b30ee4-fe4c-4d54-8b20-862a5b238eae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:17.016131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93f07c-c654-429d-81c7-48f359afd2c9 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-46b30ee4-fe4c-4d54-8b20-862a5b238eae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:17.025329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:17.026881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:17.029209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 256/543] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:17.040671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b30ee4-fe4c-4d54-8b20-862a5b238eae None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:17.206708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:17.229859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b30ee4-fe4c-4d54-8b20-862a5b238eae None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:17.272269 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d1890-b535-4ec2-bcde-1af7a94d677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:17.273394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d1890-b535-4ec2-bcde-1af7a94d677e None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:17.273906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d1890-b535-4ec2-bcde-1af7a94d677e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:17.274411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d1890-b535-4ec2-bcde-1af7a94d677e None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:17.275364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d1890-b535-4ec2-bcde-1af7a94d677e None admin] RBAC: Adding query filter params (name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:17.275865 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d1890-b535-4ec2-bcde-1af7a94d677e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:17.278057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d1890-b535-4ec2-bcde-1af7a94d677e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:17.292166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 257/544] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:17.294938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d1890-b535-4ec2-bcde-1af7a94d677e None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:17.323598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93f07c-c654-429d-81c7-48f359afd2c9 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:17.364467 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:17.419565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d1890-b535-4ec2-bcde-1af7a94d677e None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:17.442133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e38d4-1640-48b5-8f08-3cd20e7e1ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:17.442820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e38d4-1640-48b5-8f08-3cd20e7e1ffe None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:17.443108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e38d4-1640-48b5-8f08-3cd20e7e1ffe None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:17.443390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e38d4-1640-48b5-8f08-3cd20e7e1ffe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:17.444312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e38d4-1640-48b5-8f08-3cd20e7e1ffe None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=fb186ee2e3c94ca7ad203c9c0e32e498) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:17.444979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e38d4-1640-48b5-8f08-3cd20e7e1ffe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:17.446569 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e38d4-1640-48b5-8f08-3cd20e7e1ffe None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:17.460543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9798b-148a-4b30-9e30-3c697c5fbd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:17.461611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 288/545] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:17 2022] GET /identity/v3/role_assignments?user.id=fb186ee2e3c94ca7ad203c9c0e32e498&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 259 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:17.462248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9798b-148a-4b30-9e30-3c697c5fbd07 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:17.462784 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9798b-148a-4b30-9e30-3c697c5fbd07 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:17.463251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9798b-148a-4b30-9e30-3c697c5fbd07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:17.477347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9798b-148a-4b30-9e30-3c697c5fbd07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:17.479229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9798b-148a-4b30-9e30-3c697c5fbd07 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-e4b9798b-148a-4b30-9e30-3c697c5fbd07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:17.500373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:17.501454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:17.502357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:17.502357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:17.502357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:17.502357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:17.507978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 258/546] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:17.511281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e38d4-1640-48b5-8f08-3cd20e7e1ffe None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:17.675764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e38d4-1640-48b5-8f08-3cd20e7e1ffe None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:17.730215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a582017-ff4c-49fc-bbbb-9c19269ceb7d None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:17.731124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a582017-ff4c-49fc-bbbb-9c19269ceb7d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:17.731826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a582017-ff4c-49fc-bbbb-9c19269ceb7d None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:17.734103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:17.737085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 259/547] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:17.749219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8273d-b3b3-40e9-8666-43d0a7846cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:17.750480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8273d-b3b3-40e9-8666-43d0a7846cd8 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:17.750934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8273d-b3b3-40e9-8666-43d0a7846cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:17.751393 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8273d-b3b3-40e9-8666-43d0a7846cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:17.752670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8273d-b3b3-40e9-8666-43d0a7846cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:17.753169 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8273d-b3b3-40e9-8666-43d0a7846cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:17.755839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8273d-b3b3-40e9-8666-43d0a7846cd8 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:17.762725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68af29-ee48-41d0-854d-84f95a8674d8 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:17.763573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68af29-ee48-41d0-854d-84f95a8674d8 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:17.764294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68af29-ee48-41d0-854d-84f95a8674d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:17.791922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ba68af29-ee48-41d0-854d-84f95a8674d8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:17.793428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 289/548] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:17.799484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ba68af29-ee48-41d0-854d-84f95a8674d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:17.823181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8273d-b3b3-40e9-8666-43d0a7846cd8 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:17.939842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 260/549] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:35:17.986890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68af29-ee48-41d0-854d-84f95a8674d8 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:18.000342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8273d-b3b3-40e9-8666-43d0a7846cd8 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:18.045727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:18.086607 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5deb53-d8c0-462a-a875-fcc10b5cbc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:18.087412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5deb53-d8c0-462a-a875-fcc10b5cbc51 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:18.087694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5deb53-d8c0-462a-a875-fcc10b5cbc51 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:18.087971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5deb53-d8c0-462a-a875-fcc10b5cbc51 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:18.114371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5deb53-d8c0-462a-a875-fcc10b5cbc51 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:18.115939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5deb53-d8c0-462a-a875-fcc10b5cbc51 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-fb5deb53-d8c0-462a-a875-fcc10b5cbc51 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:18.132267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:18.134606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:18.136168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:18.136168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:18.136168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:18.136168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:18.136168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:18.136168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:18.136168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:18.136168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:18.136168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:18.136168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:18.136168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 290/550] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:18.153195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5deb53-d8c0-462a-a875-fcc10b5cbc51 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:18.171645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68af29-ee48-41d0-854d-84f95a8674d8 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:18.210762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2040d-3162-43a2-8c5d-505e07c5fe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:18.211571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2040d-3162-43a2-8c5d-505e07c5fe78 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:18.211852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2040d-3162-43a2-8c5d-505e07c5fe78 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:18.212123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2040d-3162-43a2-8c5d-505e07c5fe78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:18.233859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2040d-3162-43a2-8c5d-505e07c5fe78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-9fc2040d-3162-43a2-8c5d-505e07c5fe78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:18.247380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:18.248332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:18.249310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:18.258726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 261/551] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:18.266807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2040d-3162-43a2-8c5d-505e07c5fe78 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:18.289282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:18.385560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5deb53-d8c0-462a-a875-fcc10b5cbc51 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:18.395894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:18.408242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2040d-3162-43a2-8c5d-505e07c5fe78 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:18.465249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f3a40-8cb5-4c84-851c-b72a3f7834ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:18.466942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f3a40-8cb5-4c84-851c-b72a3f7834ac None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:18.467730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f3a40-8cb5-4c84-851c-b72a3f7834ac None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:18.468472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f3a40-8cb5-4c84-851c-b72a3f7834ac None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:18.470145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f3a40-8cb5-4c84-851c-b72a3f7834ac None admin] RBAC: Adding query filter params (name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:18.470939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f3a40-8cb5-4c84-851c-b72a3f7834ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:18.473774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f3a40-8cb5-4c84-851c-b72a3f7834ac None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:18.497649 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 262/552] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:18.500116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f3a40-8cb5-4c84-851c-b72a3f7834ac None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:18.514408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07578e-9376-4e4d-986c-40a9b1ab6e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:18.515432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07578e-9376-4e4d-986c-40a9b1ab6e0f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:18.515865 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07578e-9376-4e4d-986c-40a9b1ab6e0f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:18.516286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07578e-9376-4e4d-986c-40a9b1ab6e0f None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:18.517672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07578e-9376-4e4d-986c-40a9b1ab6e0f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:18.518150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07578e-9376-4e4d-986c-40a9b1ab6e0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:18.520648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07578e-9376-4e4d-986c-40a9b1ab6e0f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:18.572576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 291/553] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jul 19 20:35:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:18.591483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07578e-9376-4e4d-986c-40a9b1ab6e0f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:18.684375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f3a40-8cb5-4c84-851c-b72a3f7834ac None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:18.726332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e66de-55fe-433b-9730-d8b59378de40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:18.728265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e66de-55fe-433b-9730-d8b59378de40 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:18.729007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e66de-55fe-433b-9730-d8b59378de40 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:18.729896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e66de-55fe-433b-9730-d8b59378de40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:18.746857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e66de-55fe-433b-9730-d8b59378de40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:18.750083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e66de-55fe-433b-9730-d8b59378de40 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-360e66de-55fe-433b-9730-d8b59378de40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:18.771673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:18.772621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:18.773497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:18.773497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:18.773497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:18.773497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:18.780590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 263/554] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:18.782188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e66de-55fe-433b-9730-d8b59378de40 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:18.815431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:18.843881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07578e-9376-4e4d-986c-40a9b1ab6e0f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:18.866000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:18.919520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e66de-55fe-433b-9730-d8b59378de40 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:18.932321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290224a-f176-4087-bdff-3fd48e3e80ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:18.933000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290224a-f176-4087-bdff-3fd48e3e80ea None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:18.933282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290224a-f176-4087-bdff-3fd48e3e80ea None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:18.934268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290224a-f176-4087-bdff-3fd48e3e80ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:18.969846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1d6eb-ef85-4574-966c-ada7926df45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:18.971033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d6eb-ef85-4574-966c-ada7926df45c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:18.971516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d6eb-ef85-4574-966c-ada7926df45c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:18.972070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d6eb-ef85-4574-966c-ada7926df45c None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:18.973104 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1d6eb-ef85-4574-966c-ada7926df45c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:18.973745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1d6eb-ef85-4574-966c-ada7926df45c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:18.975627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1d6eb-ef85-4574-966c-ada7926df45c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:18.993565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 264/555] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:18.995626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1d6eb-ef85-4574-966c-ada7926df45c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:19.076740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290224a-f176-4087-bdff-3fd48e3e80ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:19.078719 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290224a-f176-4087-bdff-3fd48e3e80ea None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:19.093197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:19.192770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1d6eb-ef85-4574-966c-ada7926df45c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:19.295346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 292/556] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 704 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:35:19.301714 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290224a-f176-4087-bdff-3fd48e3e80ea None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:19.309600 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccfa2c-b9eb-4550-bf7c-731a90e59f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:19.310947 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccfa2c-b9eb-4550-bf7c-731a90e59f29 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:19.311436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccfa2c-b9eb-4550-bf7c-731a90e59f29 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:19.313384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccfa2c-b9eb-4550-bf7c-731a90e59f29 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:19.329320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ccfa2c-b9eb-4550-bf7c-731a90e59f29 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:19.331691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:19.333726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ccfa2c-b9eb-4550-bf7c-731a90e59f29 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-e1ccfa2c-b9eb-4550-bf7c-731a90e59f29 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:19.368985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:19.372239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:19.374352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:19.374352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:19.374352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:19.374352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:19.374352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:19.374352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:19.374352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:19.374352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:19.374352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:19.374352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:19.379447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 265/557] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jul 19 20:35:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:19.386995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccfa2c-b9eb-4550-bf7c-731a90e59f29 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:19.536882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290224a-f176-4087-bdff-3fd48e3e80ea None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:19.611935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590b7e7-58c6-4c0a-9cbc-c1abd8a51848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:19.613160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590b7e7-58c6-4c0a-9cbc-c1abd8a51848 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:19.613669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590b7e7-58c6-4c0a-9cbc-c1abd8a51848 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:19.614157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590b7e7-58c6-4c0a-9cbc-c1abd8a51848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:19.650126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590b7e7-58c6-4c0a-9cbc-c1abd8a51848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:19.652831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590b7e7-58c6-4c0a-9cbc-c1abd8a51848 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-b590b7e7-58c6-4c0a-9cbc-c1abd8a51848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:19.703285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:19.704941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:19.706837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:19.706837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:19.706837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:19.706837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:19.706837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 293/558] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:19.706837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e9d8d-765b-4175-b3b4-57abf884d85a None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:19.706837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e9d8d-765b-4175-b3b4-57abf884d85a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:19.706837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e9d8d-765b-4175-b3b4-57abf884d85a None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:19.710421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 294/559] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:19.714329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e9d8d-765b-4175-b3b4-57abf884d85a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:19.726795 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccfa2c-b9eb-4550-bf7c-731a90e59f29 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:19.769646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:19.784411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea66750-c264-4a48-bb6f-5f1c78b2ffcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:19.785336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66750-c264-4a48-bb6f-5f1c78b2ffcf None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:19.785786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66750-c264-4a48-bb6f-5f1c78b2ffcf None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:19.786204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66750-c264-4a48-bb6f-5f1c78b2ffcf None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:19.787265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea66750-c264-4a48-bb6f-5f1c78b2ffcf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:19.787770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea66750-c264-4a48-bb6f-5f1c78b2ffcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:19.789376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea66750-c264-4a48-bb6f-5f1c78b2ffcf None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:19.815203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 266/560] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jul 19 20:35:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:19.816922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d58f97-33de-4428-b331-9cb72c9ff90c None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:19.817406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d58f97-33de-4428-b331-9cb72c9ff90c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:19.817826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d58f97-33de-4428-b331-9cb72c9ff90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:19.834446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:19.852399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-20d58f97-33de-4428-b331-9cb72c9ff90c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:19.853749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-20d58f97-33de-4428-b331-9cb72c9ff90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:19.899034 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e9d8d-765b-4175-b3b4-57abf884d85a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:19.935109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 267/561] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:19.936019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d58f97-33de-4428-b331-9cb72c9ff90c None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:19.945421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c8318-ddbb-4617-bde0-489c22da276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:19.946107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c8318-ddbb-4617-bde0-489c22da276b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:19.946474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c8318-ddbb-4617-bde0-489c22da276b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:19.946829 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c8318-ddbb-4617-bde0-489c22da276b None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:19.947574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c8318-ddbb-4617-bde0-489c22da276b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:19.947880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c8318-ddbb-4617-bde0-489c22da276b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:19.950093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c8318-ddbb-4617-bde0-489c22da276b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:19.971247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 295/562] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:19.973618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e00c0-68fe-4796-bd07-8c0ee6e967f9 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:19.974043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e00c0-68fe-4796-bd07-8c0ee6e967f9 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:19.974328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e00c0-68fe-4796-bd07-8c0ee6e967f9 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:19.981405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 296/563] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:19.981405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e00c0-68fe-4796-bd07-8c0ee6e967f9 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:20.092779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d58f97-33de-4428-b331-9cb72c9ff90c None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:20.115061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:20.130361 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e00c0-68fe-4796-bd07-8c0ee6e967f9 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:20.149869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031084e-ee70-4bca-813a-f79125bef2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:20.150753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031084e-ee70-4bca-813a-f79125bef2fd None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.151152 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031084e-ee70-4bca-813a-f79125bef2fd None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.151543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031084e-ee70-4bca-813a-f79125bef2fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.164005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031084e-ee70-4bca-813a-f79125bef2fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:20.165619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031084e-ee70-4bca-813a-f79125bef2fd None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:20.166588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:20.184677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd272b43-05a5-467c-9635-c1d320503dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-1031084e-ee70-4bca-813a-f79125bef2fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:20.185132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:20.186169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:20.192399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:20.192399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:20.192399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:20.192399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:20.192620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272b43-05a5-467c-9635-c1d320503dfa None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.192620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272b43-05a5-467c-9635-c1d320503dfa None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.192620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272b43-05a5-467c-9635-c1d320503dfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.192808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 268/564] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:20.192808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3fa89-aedd-4330-abc5-611ebb1eac25 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.192808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3fa89-aedd-4330-abc5-611ebb1eac25 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.192808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3fa89-aedd-4330-abc5-611ebb1eac25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.203257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd272b43-05a5-467c-9635-c1d320503dfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:20.205381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd272b43-05a5-467c-9635-c1d320503dfa None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:20.208910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-f8c3fa89-aedd-4330-abc5-611ebb1eac25 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:20.210136 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-f8c3fa89-aedd-4330-abc5-611ebb1eac25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-dd272b43-05a5-467c-9635-c1d320503dfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:20.223265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:20.224306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:20.225218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:20.225218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:20.225218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:20.225218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:20.225218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:20.225218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:20.225218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:20.225218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:20.225218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 297/565] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:20.225901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd272b43-05a5-467c-9635-c1d320503dfa None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:20.305409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 269/566] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:20.305409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3fa89-aedd-4330-abc5-611ebb1eac25 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:20.356064 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd272b43-05a5-467c-9635-c1d320503dfa None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:20.406630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debb9c6-2fbe-4837-b64a-db7aed7a0d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:20.407635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debb9c6-2fbe-4837-b64a-db7aed7a0d34 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.408046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debb9c6-2fbe-4837-b64a-db7aed7a0d34 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.408367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debb9c6-2fbe-4837-b64a-db7aed7a0d34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.423643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debb9c6-2fbe-4837-b64a-db7aed7a0d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:20.432063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-5debb9c6-2fbe-4837-b64a-db7aed7a0d34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:20.432469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:20.434516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 298/567] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:20.435896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debb9c6-2fbe-4837-b64a-db7aed7a0d34 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:20.445810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3fa89-aedd-4330-abc5-611ebb1eac25 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:20.483937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9056dc-57b1-4079-ab8c-37bb4c9ff550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:20.484552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:20.484709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9056dc-57b1-4079-ab8c-37bb4c9ff550 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.484869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9056dc-57b1-4079-ab8c-37bb4c9ff550 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.485140 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9056dc-57b1-4079-ab8c-37bb4c9ff550 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.485971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9056dc-57b1-4079-ab8c-37bb4c9ff550 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:20.486287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9056dc-57b1-4079-ab8c-37bb4c9ff550 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:20.487695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9056dc-57b1-4079-ab8c-37bb4c9ff550 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:20.499976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 270/568] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:20.500595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9056dc-57b1-4079-ab8c-37bb4c9ff550 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:20.553754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debb9c6-2fbe-4837-b64a-db7aed7a0d34 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:20.594891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe04eebe-957c-4dd1-ae91-22c394a2d6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:20.595555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04eebe-957c-4dd1-ae91-22c394a2d6f8 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.595845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04eebe-957c-4dd1-ae91-22c394a2d6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.596128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe04eebe-957c-4dd1-ae91-22c394a2d6f8 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.596930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04eebe-957c-4dd1-ae91-22c394a2d6f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:20.597247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04eebe-957c-4dd1-ae91-22c394a2d6f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:20.598746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe04eebe-957c-4dd1-ae91-22c394a2d6f8 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:20.610655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9056dc-57b1-4079-ab8c-37bb4c9ff550 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:20.611278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 299/569] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:20.612581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22072879-c22a-40dc-8703-9c9444676cfe None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.612871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22072879-c22a-40dc-8703-9c9444676cfe None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.613145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22072879-c22a-40dc-8703-9c9444676cfe None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.614514 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 300/570] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:20.615118 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22072879-c22a-40dc-8703-9c9444676cfe None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:20.652627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13c750-f12b-4c78-af0f-5399cdad48f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:20.653469 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13c750-f12b-4c78-af0f-5399cdad48f0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.653751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13c750-f12b-4c78-af0f-5399cdad48f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.654031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13c750-f12b-4c78-af0f-5399cdad48f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.666737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13c750-f12b-4c78-af0f-5399cdad48f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-ea13c750-f12b-4c78-af0f-5399cdad48f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:20.674829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:20.676155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 271/571] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:20.677854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13c750-f12b-4c78-af0f-5399cdad48f0 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:20.695014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:20.728045 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22072879-c22a-40dc-8703-9c9444676cfe None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:20.729750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:20.765679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77975caa-6ab0-4c07-be96-352c5d7ef9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:20.766449 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77975caa-6ab0-4c07-be96-352c5d7ef9bf None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.766657 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77975caa-6ab0-4c07-be96-352c5d7ef9bf None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.766937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77975caa-6ab0-4c07-be96-352c5d7ef9bf None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.767937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77975caa-6ab0-4c07-be96-352c5d7ef9bf None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:20.768077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77975caa-6ab0-4c07-be96-352c5d7ef9bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:20.769317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77975caa-6ab0-4c07-be96-352c5d7ef9bf None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:20.777572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 301/572] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:20.778375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77975caa-6ab0-4c07-be96-352c5d7ef9bf None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:20.794980 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13c750-f12b-4c78-af0f-5399cdad48f0 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:20.841152 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d19c74-c1fd-4266-b9d1-d8322bbeb004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:20.842109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d19c74-c1fd-4266-b9d1-d8322bbeb004 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.842398 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d19c74-c1fd-4266-b9d1-d8322bbeb004 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.842677 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d19c74-c1fd-4266-b9d1-d8322bbeb004 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.850963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d19c74-c1fd-4266-b9d1-d8322bbeb004 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:20.852466 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d19c74-c1fd-4266-b9d1-d8322bbeb004 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-52d19c74-c1fd-4266-b9d1-d8322bbeb004 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:20.868171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:20.869194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:20.870088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:20.870088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:20.870088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:20.870088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:20.870088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:20.870088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:20.870088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:20.870088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:20.870088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 272/573] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:20.870571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3378712-e0da-4ba0-8d31-08ecad5e50ef None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.871627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3378712-e0da-4ba0-8d31-08ecad5e50ef None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.871730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3378712-e0da-4ba0-8d31-08ecad5e50ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.890870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-d3378712-e0da-4ba0-8d31-08ecad5e50ef None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:20.892790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d3378712-e0da-4ba0-8d31-08ecad5e50ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:20.907105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77975caa-6ab0-4c07-be96-352c5d7ef9bf None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:20.943984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c25ea9-4c00-4717-944f-33bdbb1cd985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:20.944477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c25ea9-4c00-4717-944f-33bdbb1cd985 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:20.944761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c25ea9-4c00-4717-944f-33bdbb1cd985 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:20.945041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c25ea9-4c00-4717-944f-33bdbb1cd985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:20.945920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c25ea9-4c00-4717-944f-33bdbb1cd985 None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=334a380fa68240c79a72746ee4a5f9dc) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:20.946235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c25ea9-4c00-4717-944f-33bdbb1cd985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:20.947669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c25ea9-4c00-4717-944f-33bdbb1cd985 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:20.953629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 302/574] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:20 2022] GET /identity/v3/role_assignments?user.id=334a380fa68240c79a72746ee4a5f9dc&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:20.954438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c25ea9-4c00-4717-944f-33bdbb1cd985 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:20.980935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 273/575] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:20.981874 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3378712-e0da-4ba0-8d31-08ecad5e50ef None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:20.995513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:21.002778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:21.122568 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c25ea9-4c00-4717-944f-33bdbb1cd985 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:21.145070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3378712-e0da-4ba0-8d31-08ecad5e50ef None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:21.166848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9ee6e-a546-47c4-86a9-43859bce47ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:21.167532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9ee6e-a546-47c4-86a9-43859bce47ed None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:21.167844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9ee6e-a546-47c4-86a9-43859bce47ed None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:21.168636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9ee6e-a546-47c4-86a9-43859bce47ed None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:21.169437 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9ee6e-a546-47c4-86a9-43859bce47ed None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:21.169755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9ee6e-a546-47c4-86a9-43859bce47ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:21.171152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9ee6e-a546-47c4-86a9-43859bce47ed None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:21.181411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 303/576] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:21.181763 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9ee6e-a546-47c4-86a9-43859bce47ed None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:21.191427 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139792d7-5bb5-48ac-ac08-bdccd6d7bdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:21.192389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139792d7-5bb5-48ac-ac08-bdccd6d7bdb5 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:21.192780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139792d7-5bb5-48ac-ac08-bdccd6d7bdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:21.193161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139792d7-5bb5-48ac-ac08-bdccd6d7bdb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:21.206450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139792d7-5bb5-48ac-ac08-bdccd6d7bdb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:21.208182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139792d7-5bb5-48ac-ac08-bdccd6d7bdb5 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-139792d7-5bb5-48ac-ac08-bdccd6d7bdb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:21.229296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:21.231276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:21.232159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:21.232159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:21.232159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:21.232159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:21.233798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 274/577] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:21.234883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139792d7-5bb5-48ac-ac08-bdccd6d7bdb5 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:21.288713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:21.300446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9ee6e-a546-47c4-86a9-43859bce47ed None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:21.303818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:21.339104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce400c18-baa4-451a-a39f-20442eb39ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:21.339801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400c18-baa4-451a-a39f-20442eb39ba4 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:21.340087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400c18-baa4-451a-a39f-20442eb39ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:21.340360 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400c18-baa4-451a-a39f-20442eb39ba4 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:21.341150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce400c18-baa4-451a-a39f-20442eb39ba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:21.341495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce400c18-baa4-451a-a39f-20442eb39ba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:21.342948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce400c18-baa4-451a-a39f-20442eb39ba4 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:21.357179 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 304/578] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:21.358135 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce400c18-baa4-451a-a39f-20442eb39ba4 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:21.359384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139792d7-5bb5-48ac-ac08-bdccd6d7bdb5 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:21.404455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b36cf-fc3c-475b-9e9e-b0720877ef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:21.406117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b36cf-fc3c-475b-9e9e-b0720877ef7d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:21.406402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b36cf-fc3c-475b-9e9e-b0720877ef7d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:21.406681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b36cf-fc3c-475b-9e9e-b0720877ef7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:21.418566 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b36cf-fc3c-475b-9e9e-b0720877ef7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-1d9b36cf-fc3c-475b-9e9e-b0720877ef7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:21.426846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:21.427801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 275/579] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:21.429585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b36cf-fc3c-475b-9e9e-b0720877ef7d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:21.491626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce400c18-baa4-451a-a39f-20442eb39ba4 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:21.534333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229106b5-af3e-4bad-abf5-9c20c10defef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:21.535306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229106b5-af3e-4bad-abf5-9c20c10defef None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:21.535604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229106b5-af3e-4bad-abf5-9c20c10defef None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:21.535993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229106b5-af3e-4bad-abf5-9c20c10defef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:21.544275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b36cf-fc3c-475b-9e9e-b0720877ef7d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:21.548181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229106b5-af3e-4bad-abf5-9c20c10defef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:21.549656 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229106b5-af3e-4bad-abf5-9c20c10defef None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:21.566048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-229106b5-af3e-4bad-abf5-9c20c10defef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:21.575773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:21.581011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:21.586846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:21.586846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:21.586846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:21.586846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:21.586846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 305/580] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:21.586846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229106b5-af3e-4bad-abf5-9c20c10defef None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:21.592037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:21.600119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af977c-59c2-4334-9a81-66aab57c337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:21.600800 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af977c-59c2-4334-9a81-66aab57c337f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:21.601205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af977c-59c2-4334-9a81-66aab57c337f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:21.601416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af977c-59c2-4334-9a81-66aab57c337f None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:21.602175 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af977c-59c2-4334-9a81-66aab57c337f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:21.602487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af977c-59c2-4334-9a81-66aab57c337f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:21.604636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af977c-59c2-4334-9a81-66aab57c337f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:21.629669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 276/581] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:21.631240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af977c-59c2-4334-9a81-66aab57c337f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:21.726181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229106b5-af3e-4bad-abf5-9c20c10defef None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:21.767797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a5aa5-d644-46c2-b8e3-8adbb7d1974b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:21.768210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a5aa5-d644-46c2-b8e3-8adbb7d1974b None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:21.768510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a5aa5-d644-46c2-b8e3-8adbb7d1974b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:21.768828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a5aa5-d644-46c2-b8e3-8adbb7d1974b None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/e1d09e168fc54f1da06b719989ae8ef0/roles/2de43a7c794c4df181f6f4c58a83a9bf` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:21.781904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af977c-59c2-4334-9a81-66aab57c337f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:21.788583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a5aa5-d644-46c2-b8e3-8adbb7d1974b None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=e1d09e168fc54f1da06b719989ae8ef0, role_id=2de43a7c794c4df181f6f4c58a83a9bf)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:21.790085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a5aa5-d644-46c2-b8e3-8adbb7d1974b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:21.817396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 306/582] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:35:21 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/e1d09e168fc54f1da06b719989ae8ef0/roles/2de43a7c794c4df181f6f4c58a83a9bf => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:35:21.817396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a5aa5-d644-46c2-b8e3-8adbb7d1974b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:21.851511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ee48b-8055-46c6-8314-1b800f3a4c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:21.852714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ee48b-8055-46c6-8314-1b800f3a4c98 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:21.853534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ee48b-8055-46c6-8314-1b800f3a4c98 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:21.854091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ee48b-8055-46c6-8314-1b800f3a4c98 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:21.856130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ee48b-8055-46c6-8314-1b800f3a4c98 None admin] RBAC: Adding query filter params (name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:21.856751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ee48b-8055-46c6-8314-1b800f3a4c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:21.859373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ee48b-8055-46c6-8314-1b800f3a4c98 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:21.880342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 277/583] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:21.886590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ee48b-8055-46c6-8314-1b800f3a4c98 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:21.937892 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:21.956951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:21.996541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a5aa5-d644-46c2-b8e3-8adbb7d1974b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:22.050871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135e190-7603-4172-b09a-3648edd7dddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:22.051590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135e190-7603-4172-b09a-3648edd7dddb None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:22.051878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135e190-7603-4172-b09a-3648edd7dddb None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:22.052160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135e190-7603-4172-b09a-3648edd7dddb None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:22.052910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135e190-7603-4172-b09a-3648edd7dddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:22.053228 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135e190-7603-4172-b09a-3648edd7dddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:22.054834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135e190-7603-4172-b09a-3648edd7dddb None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:22.067904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 307/584] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:22.068778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135e190-7603-4172-b09a-3648edd7dddb None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:22.076256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ee48b-8055-46c6-8314-1b800f3a4c98 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:22.143051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0a331-568f-4dde-b6b9-667812f387fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:22.144581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a331-568f-4dde-b6b9-667812f387fa None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:22.145805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a331-568f-4dde-b6b9-667812f387fa None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:22.146301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0a331-568f-4dde-b6b9-667812f387fa None admin] PATH_INFO: `/v3/users/glance` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:22.162742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0a331-568f-4dde-b6b9-667812f387fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:22.165073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0a331-568f-4dde-b6b9-667812f387fa None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-b5d0a331-568f-4dde-b6b9-667812f387fa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:22.185212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:22.186552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 278/585] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:22.188133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0a331-568f-4dde-b6b9-667812f387fa None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:22.198658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135e190-7603-4172-b09a-3648edd7dddb None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:22.216391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:22.243750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ad870-de73-4c7d-817f-364f2b0d13a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:22.245271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad870-de73-4c7d-817f-364f2b0d13a1 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:22.246541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad870-de73-4c7d-817f-364f2b0d13a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:22.247040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad870-de73-4c7d-817f-364f2b0d13a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:22.266533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ad870-de73-4c7d-817f-364f2b0d13a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:22.268373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ad870-de73-4c7d-817f-364f2b0d13a1 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-fc4ad870-de73-4c7d-817f-364f2b0d13a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:22.292043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:22.293618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:22.294632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:22.294632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:22.294632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:22.294632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:22.297312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 308/586] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:22.302988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ad870-de73-4c7d-817f-364f2b0d13a1 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:22.324376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:22.329953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0a331-568f-4dde-b6b9-667812f387fa None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:22.367381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5161d-0e31-4ec9-aa53-e2e572fad7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:22.368192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5161d-0e31-4ec9-aa53-e2e572fad7bd None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:22.368484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5161d-0e31-4ec9-aa53-e2e572fad7bd None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:22.368794 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5161d-0e31-4ec9-aa53-e2e572fad7bd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:22.378725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5161d-0e31-4ec9-aa53-e2e572fad7bd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:22.380217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5161d-0e31-4ec9-aa53-e2e572fad7bd None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-cda5161d-0e31-4ec9-aa53-e2e572fad7bd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:22.396693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:22.401489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 279/587] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:22.403445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5161d-0e31-4ec9-aa53-e2e572fad7bd None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:22.477979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ad870-de73-4c7d-817f-364f2b0d13a1 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:22.515908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c06c5-cfa1-49f2-8aa3-6045b758eb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:22.516942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c06c5-cfa1-49f2-8aa3-6045b758eb27 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:22.517501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c06c5-cfa1-49f2-8aa3-6045b758eb27 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:22.518075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c06c5-cfa1-49f2-8aa3-6045b758eb27 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:22.519815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c06c5-cfa1-49f2-8aa3-6045b758eb27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:22.520510 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5161d-0e31-4ec9-aa53-e2e572fad7bd None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:22.521109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c06c5-cfa1-49f2-8aa3-6045b758eb27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:22.523807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c06c5-cfa1-49f2-8aa3-6045b758eb27 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:22.544451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 309/588] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jul 19 20:35:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:22.545703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c06c5-cfa1-49f2-8aa3-6045b758eb27 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:22.566067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2d674-33ab-471c-9572-3048211718fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:22.566628 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2d674-33ab-471c-9572-3048211718fe None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:22.566927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2d674-33ab-471c-9572-3048211718fe None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:22.567188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2d674-33ab-471c-9572-3048211718fe None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:22.567906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2d674-33ab-471c-9572-3048211718fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:22.568214 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2d674-33ab-471c-9572-3048211718fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:22.569624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2d674-33ab-471c-9572-3048211718fe None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:22.589415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 280/589] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:22.589415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2d674-33ab-471c-9572-3048211718fe None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:22.604611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:22.616143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:22.733288 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2d674-33ab-471c-9572-3048211718fe None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:22.756835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c06c5-cfa1-49f2-8aa3-6045b758eb27 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:22.772685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460e385-ad97-49cb-8a69-8e43022bc213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:22.773522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460e385-ad97-49cb-8a69-8e43022bc213 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:22.773803 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460e385-ad97-49cb-8a69-8e43022bc213 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:22.774076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460e385-ad97-49cb-8a69-8e43022bc213 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:22.787042 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460e385-ad97-49cb-8a69-8e43022bc213 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:22.788502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460e385-ad97-49cb-8a69-8e43022bc213 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:22.800482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dd970-0a63-4cd4-b940-6337dc016fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:22.801566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dd970-0a63-4cd4-b940-6337dc016fd5 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:22.802062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dd970-0a63-4cd4-b940-6337dc016fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:22.802548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dd970-0a63-4cd4-b940-6337dc016fd5 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:22.803767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dd970-0a63-4cd4-b940-6337dc016fd5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:22.804300 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dd970-0a63-4cd4-b940-6337dc016fd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:22.806165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dd970-0a63-4cd4-b940-6337dc016fd5 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-1460e385-ad97-49cb-8a69-8e43022bc213 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:22.812006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:22.813103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:22.814045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:22.814045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:22.814045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:22.814045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:22.818265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 281/590] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:22.819150 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460e385-ad97-49cb-8a69-8e43022bc213 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:22.831123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 310/591] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jul 19 20:35:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:22.832346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dd970-0a63-4cd4-b940-6337dc016fd5 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:22.908221 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:22.943158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:22.947699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460e385-ad97-49cb-8a69-8e43022bc213 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:22.973223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dd970-0a63-4cd4-b940-6337dc016fd5 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:22.987474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3f2cd-9062-4394-9f5b-75ddfec7b687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:22.988386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3f2cd-9062-4394-9f5b-75ddfec7b687 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:22.988684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3f2cd-9062-4394-9f5b-75ddfec7b687 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:22.989004 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3f2cd-9062-4394-9f5b-75ddfec7b687 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:22.997991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3f2cd-9062-4394-9f5b-75ddfec7b687 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:22.999578 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3f2cd-9062-4394-9f5b-75ddfec7b687 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-f4d3f2cd-9062-4394-9f5b-75ddfec7b687 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:23.018752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:23.019770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:23.020716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:23.020716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:23.020716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:23.020716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:23.020716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:23.020716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:23.020716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:23.020716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:23.020716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:35:23.020716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:23.025082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 282/592] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:23.026119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3f2cd-9062-4394-9f5b-75ddfec7b687 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:23.029493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3265c-d26a-4e19-af75-32e2d96cc801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:23.030703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3265c-d26a-4e19-af75-32e2d96cc801 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:23.031243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3265c-d26a-4e19-af75-32e2d96cc801 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:23.031744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3265c-d26a-4e19-af75-32e2d96cc801 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:23.032757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3265c-d26a-4e19-af75-32e2d96cc801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:23.033434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3265c-d26a-4e19-af75-32e2d96cc801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:23.035114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3265c-d26a-4e19-af75-32e2d96cc801 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:23.053192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 311/593] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:23.054528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3265c-d26a-4e19-af75-32e2d96cc801 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:23.163338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3f2cd-9062-4394-9f5b-75ddfec7b687 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:23.179823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3265c-d26a-4e19-af75-32e2d96cc801 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:23.196778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:23.206599 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a7ef9-6376-46f7-ab35-1a99113dca05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:23.207896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a7ef9-6376-46f7-ab35-1a99113dca05 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:23.208665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a7ef9-6376-46f7-ab35-1a99113dca05 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:23.209218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a7ef9-6376-46f7-ab35-1a99113dca05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:23.222278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1702ec-1647-4a64-934d-a03fc3eb795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:23.223215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1702ec-1647-4a64-934d-a03fc3eb795c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:23.223465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a7ef9-6376-46f7-ab35-1a99113dca05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:23.223583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1702ec-1647-4a64-934d-a03fc3eb795c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:23.223981 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1702ec-1647-4a64-934d-a03fc3eb795c None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:23.225175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1702ec-1647-4a64-934d-a03fc3eb795c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:23.225516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1702ec-1647-4a64-934d-a03fc3eb795c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:23.226526 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a7ef9-6376-46f7-ab35-1a99113dca05 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:23.227188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1702ec-1647-4a64-934d-a03fc3eb795c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:23.241408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-768a7ef9-6376-46f7-ab35-1a99113dca05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:23.250310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:23.251322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:23.252196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:23.252196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:23.252196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:23.252196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:23.254437 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 283/594] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:23.255800 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a7ef9-6376-46f7-ab35-1a99113dca05 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:23.262088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 312/595] 10.222.0.70 () {56 vars in 1139 bytes} [Tue Jul 19 20:35:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:23.264511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1702ec-1647-4a64-934d-a03fc3eb795c None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:23.401325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a7ef9-6376-46f7-ab35-1a99113dca05 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:23.407955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1702ec-1647-4a64-934d-a03fc3eb795c None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:23.445576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319401b-ac4a-4f6d-8c28-c48eb217cca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:23.446727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319401b-ac4a-4f6d-8c28-c48eb217cca9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:23.447011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319401b-ac4a-4f6d-8c28-c48eb217cca9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:23.447289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319401b-ac4a-4f6d-8c28-c48eb217cca9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:23.456674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e4268-53c5-4a13-b338-87fcd569213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:23.457452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e4268-53c5-4a13-b338-87fcd569213b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:23.457652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e4268-53c5-4a13-b338-87fcd569213b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:23.457938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e4268-53c5-4a13-b338-87fcd569213b None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:23.458666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e4268-53c5-4a13-b338-87fcd569213b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:23.458973 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e4268-53c5-4a13-b338-87fcd569213b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:23.459737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319401b-ac4a-4f6d-8c28-c48eb217cca9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:23.460334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e4268-53c5-4a13-b338-87fcd569213b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:23.462896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319401b-ac4a-4f6d-8c28-c48eb217cca9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-4319401b-ac4a-4f6d-8c28-c48eb217cca9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:23.479145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:23.480300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:23.481443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:23.481443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:23.481443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:23.481443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:23.481443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:23.481443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:23.481443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:23.481443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:23.483294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 284/596] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:23.486827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 313/597] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:23.488082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e4268-53c5-4a13-b338-87fcd569213b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:23.489425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eb908-0ddb-4494-95d6-41f8ba4e2f63 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:23.489727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eb908-0ddb-4494-95d6-41f8ba4e2f63 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:23.490015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eb908-0ddb-4494-95d6-41f8ba4e2f63 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:23.497414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 285/598] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:23.497414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711eb908-0ddb-4494-95d6-41f8ba4e2f63 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:23.550108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:23.563532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:23.617990 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e4268-53c5-4a13-b338-87fcd569213b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:23.626308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711eb908-0ddb-4494-95d6-41f8ba4e2f63 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:23.670231 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62e02e-b5fc-46dd-9004-6aca9c6dd3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:23.671229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62e02e-b5fc-46dd-9004-6aca9c6dd3a8 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:23.671665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62e02e-b5fc-46dd-9004-6aca9c6dd3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:23.672140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62e02e-b5fc-46dd-9004-6aca9c6dd3a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:23.673464 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f416d-acac-4373-9c90-d793d769c89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:23.674414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f416d-acac-4373-9c90-d793d769c89d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:23.674883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f416d-acac-4373-9c90-d793d769c89d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:23.675493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f416d-acac-4373-9c90-d793d769c89d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:23.684595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f416d-acac-4373-9c90-d793d769c89d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:23.686651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f416d-acac-4373-9c90-d793d769c89d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:23.688666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62e02e-b5fc-46dd-9004-6aca9c6dd3a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:23.690515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62e02e-b5fc-46dd-9004-6aca9c6dd3a8 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-e09f416d-acac-4373-9c90-d793d769c89d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:23.704524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:23.705447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:23.706348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:23.706348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:23.706348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:23.706348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:23.706348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:23.706348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:23.706348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:23.706348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:23.712700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 286/599] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:23.712700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f416d-acac-4373-9c90-d793d769c89d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-4b62e02e-b5fc-46dd-9004-6aca9c6dd3a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:23.713770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:23.714648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:23.722009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:23.722009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:23.722009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:23.722009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:23.722009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 314/600] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:23.722009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48231a74-3d6a-414d-8052-1731ac46eba7 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:23.722009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48231a74-3d6a-414d-8052-1731ac46eba7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:23.722009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48231a74-3d6a-414d-8052-1731ac46eba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:23.742147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-48231a74-3d6a-414d-8052-1731ac46eba7 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:23.743391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-48231a74-3d6a-414d-8052-1731ac46eba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:23.833232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 315/601] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:23.834654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48231a74-3d6a-414d-8052-1731ac46eba7 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:23.844527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f416d-acac-4373-9c90-d793d769c89d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:23.846413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:23.890855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612358a-91e7-4a9c-ad9e-f316c932b315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:23.891659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612358a-91e7-4a9c-ad9e-f316c932b315 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:23.892061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612358a-91e7-4a9c-ad9e-f316c932b315 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:23.892416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612358a-91e7-4a9c-ad9e-f316c932b315 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:23.893292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612358a-91e7-4a9c-ad9e-f316c932b315 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:23.893714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612358a-91e7-4a9c-ad9e-f316c932b315 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:23.895240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612358a-91e7-4a9c-ad9e-f316c932b315 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:23.900776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:23.919067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 287/602] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:23.921186 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612358a-91e7-4a9c-ad9e-f316c932b315 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:23.983129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48231a74-3d6a-414d-8052-1731ac46eba7 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:24.038670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4680067-5b8b-4993-a467-4ce2cb5a7350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:24.039626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4680067-5b8b-4993-a467-4ce2cb5a7350 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:24.040057 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4680067-5b8b-4993-a467-4ce2cb5a7350 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:24.040542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4680067-5b8b-4993-a467-4ce2cb5a7350 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:24.041349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4680067-5b8b-4993-a467-4ce2cb5a7350 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:24.041685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4680067-5b8b-4993-a467-4ce2cb5a7350 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:24.043162 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4680067-5b8b-4993-a467-4ce2cb5a7350 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:24.061198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 316/603] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:24.062310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4680067-5b8b-4993-a467-4ce2cb5a7350 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:24.063442 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612358a-91e7-4a9c-ad9e-f316c932b315 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:24.117061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498ea50-31ee-48ee-a11e-509ae7631c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:24.117806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498ea50-31ee-48ee-a11e-509ae7631c4e None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:24.118257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498ea50-31ee-48ee-a11e-509ae7631c4e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:24.118879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498ea50-31ee-48ee-a11e-509ae7631c4e None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:24.120065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498ea50-31ee-48ee-a11e-509ae7631c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:24.120624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498ea50-31ee-48ee-a11e-509ae7631c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:24.122825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498ea50-31ee-48ee-a11e-509ae7631c4e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:24.135959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 288/604] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:24.141640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498ea50-31ee-48ee-a11e-509ae7631c4e None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:24.168213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:24.189910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:24.202973 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4680067-5b8b-4993-a467-4ce2cb5a7350 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:24.255793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854b57f-2b01-4a80-8ad1-dad0bf1123ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:24.256652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854b57f-2b01-4a80-8ad1-dad0bf1123ae None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:24.256935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854b57f-2b01-4a80-8ad1-dad0bf1123ae None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:24.257215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854b57f-2b01-4a80-8ad1-dad0bf1123ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:24.271802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498ea50-31ee-48ee-a11e-509ae7631c4e None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:24.272250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854b57f-2b01-4a80-8ad1-dad0bf1123ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-a854b57f-2b01-4a80-8ad1-dad0bf1123ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:24.281487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:24.283678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 317/605] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:24.285137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854b57f-2b01-4a80-8ad1-dad0bf1123ae None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:24.323646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9ff27-d997-4710-ab3d-eb3adb516626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:24.325406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9ff27-d997-4710-ab3d-eb3adb516626 None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:24.325689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9ff27-d997-4710-ab3d-eb3adb516626 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:24.325999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9ff27-d997-4710-ab3d-eb3adb516626 None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/82e1192b2d554199a08debb4aa392479/roles/2de43a7c794c4df181f6f4c58a83a9bf` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:24.342884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9ff27-d997-4710-ab3d-eb3adb516626 None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=82e1192b2d554199a08debb4aa392479, role_id=2de43a7c794c4df181f6f4c58a83a9bf)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:24.345166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9ff27-d997-4710-ab3d-eb3adb516626 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:24.360877 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 289/606] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:35:24 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/82e1192b2d554199a08debb4aa392479/roles/2de43a7c794c4df181f6f4c58a83a9bf => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:35:24.362034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9ff27-d997-4710-ab3d-eb3adb516626 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:24.401817 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854b57f-2b01-4a80-8ad1-dad0bf1123ae None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:24.450189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ece247-22fc-47ea-b982-fb7c4b345194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:24.451102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ece247-22fc-47ea-b982-fb7c4b345194 None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:24.451387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ece247-22fc-47ea-b982-fb7c4b345194 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:24.451668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ece247-22fc-47ea-b982-fb7c4b345194 None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/a75951e9512c420992a4fd6c1a6b1176/roles/2de43a7c794c4df181f6f4c58a83a9bf` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:24.474418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ece247-22fc-47ea-b982-fb7c4b345194 None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=a75951e9512c420992a4fd6c1a6b1176, role_id=2de43a7c794c4df181f6f4c58a83a9bf)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:24.476541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ece247-22fc-47ea-b982-fb7c4b345194 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:24.480777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:24.491991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 318/607] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:35:24 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/a75951e9512c420992a4fd6c1a6b1176/roles/2de43a7c794c4df181f6f4c58a83a9bf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:35:24.493049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ece247-22fc-47ea-b982-fb7c4b345194 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:24.510044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:24.550210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9ff27-d997-4710-ab3d-eb3adb516626 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:24.616960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bad11a-411f-4176-844f-1e5c30b15af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:24.617932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bad11a-411f-4176-844f-1e5c30b15af4 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:24.618560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bad11a-411f-4176-844f-1e5c30b15af4 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:24.618951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bad11a-411f-4176-844f-1e5c30b15af4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:24.633509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bad11a-411f-4176-844f-1e5c30b15af4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:24.635242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bad11a-411f-4176-844f-1e5c30b15af4 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-55bad11a-411f-4176-844f-1e5c30b15af4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:24.655862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:24.657190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:24.658637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:24.658637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:24.658637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:24.658637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:24.658637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:24.658637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:24.658637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:24.658637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:24.658637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 290/608] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:24.659331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15233c86-d25c-444c-a946-e3138b5ebe74 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:24.659331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15233c86-d25c-444c-a946-e3138b5ebe74 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:24.659331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15233c86-d25c-444c-a946-e3138b5ebe74 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:24.666041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 291/609] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:24.668833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15233c86-d25c-444c-a946-e3138b5ebe74 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:24.670072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ece247-22fc-47ea-b982-fb7c4b345194 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:24.733701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e209fb-f561-416d-b68c-e14cf6073928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:24.735114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e209fb-f561-416d-b68c-e14cf6073928 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:24.735643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e209fb-f561-416d-b68c-e14cf6073928 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:24.736106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e209fb-f561-416d-b68c-e14cf6073928 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:24.737104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e209fb-f561-416d-b68c-e14cf6073928 None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:24.737777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e209fb-f561-416d-b68c-e14cf6073928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:24.739176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e209fb-f561-416d-b68c-e14cf6073928 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:24.752714 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 319/610] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:24.758840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a5e9e-a189-4aca-a51b-cc68eee322e7 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:24.760292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a5e9e-a189-4aca-a51b-cc68eee322e7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:24.760842 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a5e9e-a189-4aca-a51b-cc68eee322e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:24.788935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-b48a5e9e-a189-4aca-a51b-cc68eee322e7 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:24.790936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b48a5e9e-a189-4aca-a51b-cc68eee322e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:24.816987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15233c86-d25c-444c-a946-e3138b5ebe74 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:24.853010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:24.860944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b95ab-a9f9-4c3c-a990-fbb8e814a5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:24.862016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b95ab-a9f9-4c3c-a990-fbb8e814a5e8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:24.862596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b95ab-a9f9-4c3c-a990-fbb8e814a5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:24.863469 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b95ab-a9f9-4c3c-a990-fbb8e814a5e8 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:24.865173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b95ab-a9f9-4c3c-a990-fbb8e814a5e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:24.865851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b95ab-a9f9-4c3c-a990-fbb8e814a5e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:24.868471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b95ab-a9f9-4c3c-a990-fbb8e814a5e8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:24.886116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 292/611] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:24.887642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b95ab-a9f9-4c3c-a990-fbb8e814a5e8 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:24.892727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 320/612] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:24.894370 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a5e9e-a189-4aca-a51b-cc68eee322e7 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:24.897176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:25.013630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b95ab-a9f9-4c3c-a990-fbb8e814a5e8 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:25.068424 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a5e9e-a189-4aca-a51b-cc68eee322e7 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:25.069152 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e3a88-4671-4cf4-aaa9-ca41eb810a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:25.070252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e3a88-4671-4cf4-aaa9-ca41eb810a82 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:25.070842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e3a88-4671-4cf4-aaa9-ca41eb810a82 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:25.071428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e3a88-4671-4cf4-aaa9-ca41eb810a82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:25.087630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e3a88-4671-4cf4-aaa9-ca41eb810a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:25.091085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e3a88-4671-4cf4-aaa9-ca41eb810a82 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-c13e3a88-4671-4cf4-aaa9-ca41eb810a82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:25.115950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:25.116915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:25.118180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:25.118180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:25.118180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:25.118180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:25.124898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 293/613] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:25.126139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e3a88-4671-4cf4-aaa9-ca41eb810a82 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:25.133187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8658b-ab93-479c-afdd-1be81aff4d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:25.134797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8658b-ab93-479c-afdd-1be81aff4d4e None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:25.135286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8658b-ab93-479c-afdd-1be81aff4d4e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:25.135793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8658b-ab93-479c-afdd-1be81aff4d4e None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/fb186ee2e3c94ca7ad203c9c0e32e498/roles/2de43a7c794c4df181f6f4c58a83a9bf` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:25.163587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8658b-ab93-479c-afdd-1be81aff4d4e None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=fb186ee2e3c94ca7ad203c9c0e32e498, role_id=2de43a7c794c4df181f6f4c58a83a9bf)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:25.166373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8658b-ab93-479c-afdd-1be81aff4d4e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:25.179744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:25.192100 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 321/614] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:35:24 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/fb186ee2e3c94ca7ad203c9c0e32e498/roles/2de43a7c794c4df181f6f4c58a83a9bf => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:35:25.193967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8658b-ab93-479c-afdd-1be81aff4d4e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:25.252556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e3a88-4671-4cf4-aaa9-ca41eb810a82 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:25.320454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5038e39-1c2b-4bb0-8d10-d86d7c5d3a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:25.322878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5038e39-1c2b-4bb0-8d10-d86d7c5d3a27 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:25.326778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5038e39-1c2b-4bb0-8d10-d86d7c5d3a27 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:25.327368 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5038e39-1c2b-4bb0-8d10-d86d7c5d3a27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:25.330400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:25.354707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5038e39-1c2b-4bb0-8d10-d86d7c5d3a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-f5038e39-1c2b-4bb0-8d10-d86d7c5d3a27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:25.384601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:25.385775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:25.386810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:25.392230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 294/615] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:25.403133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5038e39-1c2b-4bb0-8d10-d86d7c5d3a27 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:25.461260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8658b-ab93-479c-afdd-1be81aff4d4e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:25.524403 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873f85d-39eb-49bd-83e7-72093565c2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:25.525563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873f85d-39eb-49bd-83e7-72093565c2e5 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:25.526074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873f85d-39eb-49bd-83e7-72093565c2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:25.526578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873f85d-39eb-49bd-83e7-72093565c2e5 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:25.528572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873f85d-39eb-49bd-83e7-72093565c2e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:25.529117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873f85d-39eb-49bd-83e7-72093565c2e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:25.531887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873f85d-39eb-49bd-83e7-72093565c2e5 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:25.555120 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 322/616] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:25.562623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873f85d-39eb-49bd-83e7-72093565c2e5 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:25.687432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5038e39-1c2b-4bb0-8d10-d86d7c5d3a27 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:25.694807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:25.762692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0fb3d-d60c-451e-b510-47034bb14b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:25.763392 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0fb3d-d60c-451e-b510-47034bb14b8d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:25.763689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0fb3d-d60c-451e-b510-47034bb14b8d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:25.763961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0fb3d-d60c-451e-b510-47034bb14b8d None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:25.764729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0fb3d-d60c-451e-b510-47034bb14b8d None admin] RBAC: Adding query filter params (name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:25.765148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0fb3d-d60c-451e-b510-47034bb14b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:25.766589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0fb3d-d60c-451e-b510-47034bb14b8d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:25.796149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 295/617] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:25.801964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0fb3d-d60c-451e-b510-47034bb14b8d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:25.809027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873f85d-39eb-49bd-83e7-72093565c2e5 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:25.867209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:25.876691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e314ee4-9bf8-414e-b861-e226d3031d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:25.878104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e314ee4-9bf8-414e-b861-e226d3031d73 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:25.878609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e314ee4-9bf8-414e-b861-e226d3031d73 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:25.879113 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e314ee4-9bf8-414e-b861-e226d3031d73 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:25.895414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e314ee4-9bf8-414e-b861-e226d3031d73 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:25.898392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e314ee4-9bf8-414e-b861-e226d3031d73 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-8e314ee4-9bf8-414e-b861-e226d3031d73 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:25.928459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:25.930004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:25.931273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:25.931273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:25.931273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:25.931273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:25.931273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:25.931273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:25.931273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:25.931273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:25.931273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:25.931273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:25.931273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 323/618] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:25.940684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e314ee4-9bf8-414e-b861-e226d3031d73 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:26.045209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0fb3d-d60c-451e-b510-47034bb14b8d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:26.092221 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db19349-c676-474a-9977-571ec5656330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:26.093466 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db19349-c676-474a-9977-571ec5656330 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:26.093976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db19349-c676-474a-9977-571ec5656330 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:26.094476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db19349-c676-474a-9977-571ec5656330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:26.125815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db19349-c676-474a-9977-571ec5656330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:26.128489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db19349-c676-474a-9977-571ec5656330 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:26.151238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e314ee4-9bf8-414e-b861-e226d3031d73 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-4db19349-c676-474a-9977-571ec5656330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:26.169446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:26.170537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:26.171414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:26.171414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:26.171414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:26.171414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:26.173922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 296/619] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:26.179220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db19349-c676-474a-9977-571ec5656330 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:26.209943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:26.213075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f80378-49f4-426f-b1cc-99b7cd50965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:26.218254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f80378-49f4-426f-b1cc-99b7cd50965b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:26.220312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f80378-49f4-426f-b1cc-99b7cd50965b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:26.220995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f80378-49f4-426f-b1cc-99b7cd50965b None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:26.224580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f80378-49f4-426f-b1cc-99b7cd50965b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:26.225447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f80378-49f4-426f-b1cc-99b7cd50965b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:26.228966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f80378-49f4-426f-b1cc-99b7cd50965b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:26.269939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 324/620] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jul 19 20:35:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:26.276621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f80378-49f4-426f-b1cc-99b7cd50965b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:26.342399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:26.451102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db19349-c676-474a-9977-571ec5656330 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:26.506355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f80378-49f4-426f-b1cc-99b7cd50965b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:26.513869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4eda8-f5e7-455a-bfa8-a7b9208186c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:26.514862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4eda8-f5e7-455a-bfa8-a7b9208186c9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:26.515355 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4eda8-f5e7-455a-bfa8-a7b9208186c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:26.515859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4eda8-f5e7-455a-bfa8-a7b9208186c9 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:26.516797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4eda8-f5e7-455a-bfa8-a7b9208186c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:26.517329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4eda8-f5e7-455a-bfa8-a7b9208186c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:26.519139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4eda8-f5e7-455a-bfa8-a7b9208186c9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:26.549449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 297/621] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:26.550632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4eda8-f5e7-455a-bfa8-a7b9208186c9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:26.577217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55e1d8-0d26-42f1-8adc-655760c81977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:26.578185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55e1d8-0d26-42f1-8adc-655760c81977 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:26.578475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55e1d8-0d26-42f1-8adc-655760c81977 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:26.578827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55e1d8-0d26-42f1-8adc-655760c81977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:26.592277 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55e1d8-0d26-42f1-8adc-655760c81977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:26.593875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55e1d8-0d26-42f1-8adc-655760c81977 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-4b55e1d8-0d26-42f1-8adc-655760c81977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:26.618006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:26.618941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:26.619825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:26.619825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:26.619825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:26.619825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:26.626106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 325/622] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:26.629073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55e1d8-0d26-42f1-8adc-655760c81977 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:26.751735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:26.805752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:26.816456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4eda8-f5e7-455a-bfa8-a7b9208186c9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:26.862967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55e1d8-0d26-42f1-8adc-655760c81977 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:26.957472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f5eba-60d7-4a21-b1b0-82a342a664eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:26.961383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f5eba-60d7-4a21-b1b0-82a342a664eb None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:26.961383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f5eba-60d7-4a21-b1b0-82a342a664eb None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:26.961383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f5eba-60d7-4a21-b1b0-82a342a664eb None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:26.961383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f5eba-60d7-4a21-b1b0-82a342a664eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:26.961383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f5eba-60d7-4a21-b1b0-82a342a664eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:26.969391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f5eba-60d7-4a21-b1b0-82a342a664eb None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:26.977391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f38e6b-b2aa-4f99-981d-8a6d53917f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:26.977391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f38e6b-b2aa-4f99-981d-8a6d53917f1d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:26.977391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f38e6b-b2aa-4f99-981d-8a6d53917f1d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:26.977762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f38e6b-b2aa-4f99-981d-8a6d53917f1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:26.980777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 326/623] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:26.989392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f5eba-60d7-4a21-b1b0-82a342a664eb None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:27.017085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f38e6b-b2aa-4f99-981d-8a6d53917f1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:27.025387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f38e6b-b2aa-4f99-981d-8a6d53917f1d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-07f38e6b-b2aa-4f99-981d-8a6d53917f1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:27.081470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:27.082894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:27.083841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:27.083841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:27.083841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:27.083841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:27.083841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:27.083841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:27.083841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:27.083841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:27.083841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:27.083841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:27.086648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 298/624] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jul 19 20:35:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 536 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:27.102528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f38e6b-b2aa-4f99-981d-8a6d53917f1d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:27.107445 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f5eba-60d7-4a21-b1b0-82a342a664eb None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:27.158933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f71f2-3ab0-4b28-bf48-34cbc47d4e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:27.160489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f71f2-3ab0-4b28-bf48-34cbc47d4e14 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:27.160840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f71f2-3ab0-4b28-bf48-34cbc47d4e14 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:27.161156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f71f2-3ab0-4b28-bf48-34cbc47d4e14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:27.172999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f71f2-3ab0-4b28-bf48-34cbc47d4e14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:27.176144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f71f2-3ab0-4b28-bf48-34cbc47d4e14 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-ef8f71f2-3ab0-4b28-bf48-34cbc47d4e14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:27.201124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:27.202158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:27.203055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:27.203055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:27.203055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:27.203055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:27.203055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:27.203055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:27.203055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:27.203055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:27.203055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 327/625] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:27.204102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf980f5-2f08-414e-9ceb-da18340cf283 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:27.204393 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf980f5-2f08-414e-9ceb-da18340cf283 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:27.204674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf980f5-2f08-414e-9ceb-da18340cf283 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:27.216569 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 328/626] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:27.216569 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf980f5-2f08-414e-9ceb-da18340cf283 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:27.222673 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:27.372898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f38e6b-b2aa-4f99-981d-8a6d53917f1d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:27.392610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf980f5-2f08-414e-9ceb-da18340cf283 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:27.431745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dbec2-1399-4dec-ae63-0677a13441b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:27.432647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dbec2-1399-4dec-ae63-0677a13441b2 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:27.433011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dbec2-1399-4dec-ae63-0677a13441b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:27.433350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dbec2-1399-4dec-ae63-0677a13441b2 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:27.434264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dbec2-1399-4dec-ae63-0677a13441b2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:27.434555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dbec2-1399-4dec-ae63-0677a13441b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:27.436011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dbec2-1399-4dec-ae63-0677a13441b2 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:27.445226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:27.462074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 299/627] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jul 19 20:35:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:27.463376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9ca34-a27b-49a9-b160-e884512b31f5 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:27.463765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9ca34-a27b-49a9-b160-e884512b31f5 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:27.464136 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9ca34-a27b-49a9-b160-e884512b31f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:27.465057 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a25b0d-cec4-452a-a468-faa569fca0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:27.467000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a25b0d-cec4-452a-a468-faa569fca0bc None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:27.467173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a25b0d-cec4-452a-a468-faa569fca0bc None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:27.467511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a25b0d-cec4-452a-a468-faa569fca0bc None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:27.469064 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a25b0d-cec4-452a-a468-faa569fca0bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:27.469684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a25b0d-cec4-452a-a468-faa569fca0bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:27.472127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a25b0d-cec4-452a-a468-faa569fca0bc None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:27.491780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-13b9ca34-a27b-49a9-b160-e884512b31f5 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:27.493832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-13b9ca34-a27b-49a9-b160-e884512b31f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:27.494802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 329/628] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:27.497189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483c63a-96e5-4a94-b1f6-e7c2ffb50ab3 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:27.497822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483c63a-96e5-4a94-b1f6-e7c2ffb50ab3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:27.498316 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483c63a-96e5-4a94-b1f6-e7c2ffb50ab3 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:27.505769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 330/629] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:27.506808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483c63a-96e5-4a94-b1f6-e7c2ffb50ab3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:27.579932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 300/630] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:27.580898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9ca34-a27b-49a9-b160-e884512b31f5 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:27.600640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:27.653915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483c63a-96e5-4a94-b1f6-e7c2ffb50ab3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:27.703626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd6bbb-cbc1-4b2c-bd39-fe6179e9b530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:27.704299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd6bbb-cbc1-4b2c-bd39-fe6179e9b530 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:27.704720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd6bbb-cbc1-4b2c-bd39-fe6179e9b530 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:27.705058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd6bbb-cbc1-4b2c-bd39-fe6179e9b530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:27.728507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9ca34-a27b-49a9-b160-e884512b31f5 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:27.759665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:27.771996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88a341-7082-462b-bfae-aa1d9027eed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:27.773189 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a341-7082-462b-bfae-aa1d9027eed8 None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:27.773923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a341-7082-462b-bfae-aa1d9027eed8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:27.774397 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88a341-7082-462b-bfae-aa1d9027eed8 None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/334a380fa68240c79a72746ee4a5f9dc/roles/2de43a7c794c4df181f6f4c58a83a9bf` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:27.796150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd6bbb-cbc1-4b2c-bd39-fe6179e9b530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:27.798665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd6bbb-cbc1-4b2c-bd39-fe6179e9b530 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:27.799529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88a341-7082-462b-bfae-aa1d9027eed8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=334a380fa68240c79a72746ee4a5f9dc, role_id=2de43a7c794c4df181f6f4c58a83a9bf)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:27.801901 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88a341-7082-462b-bfae-aa1d9027eed8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:27.822614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 301/631] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:35:27 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/334a380fa68240c79a72746ee4a5f9dc/roles/2de43a7c794c4df181f6f4c58a83a9bf => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:35:27.824852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a669b-e230-499e-8317-b596e2ad1097 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:27.825232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a669b-e230-499e-8317-b596e2ad1097 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:27.825623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a669b-e230-499e-8317-b596e2ad1097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:27.862265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ad7a669b-e230-499e-8317-b596e2ad1097 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:27.863773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ad7a669b-e230-499e-8317-b596e2ad1097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:27.926940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:27.984223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 302/632] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:27.985534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a669b-e230-499e-8317-b596e2ad1097 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:28.000106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 331/633] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:28.002972 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42d89e-9507-4700-a89a-94a5e9ce2e92 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:28.003841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42d89e-9507-4700-a89a-94a5e9ce2e92 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:28.004602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42d89e-9507-4700-a89a-94a5e9ce2e92 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:28.007934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 332/634] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:28.009516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42d89e-9507-4700-a89a-94a5e9ce2e92 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:28.129666 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a669b-e230-499e-8317-b596e2ad1097 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:28.144080 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:28.149840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42d89e-9507-4700-a89a-94a5e9ce2e92 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:28.181761 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e62c18-9d3f-419b-8860-10d9a164d26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:28.182923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62c18-9d3f-419b-8860-10d9a164d26a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:28.183922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62c18-9d3f-419b-8860-10d9a164d26a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:28.184398 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62c18-9d3f-419b-8860-10d9a164d26a None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:28.199107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e62c18-9d3f-419b-8860-10d9a164d26a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-b2e62c18-9d3f-419b-8860-10d9a164d26a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:28.207730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:28.208857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 303/635] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:28.215049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8633ac-57b8-4a3f-b670-4819f89fe8e9 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:28.216345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8633ac-57b8-4a3f-b670-4819f89fe8e9 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:28.216345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8633ac-57b8-4a3f-b670-4819f89fe8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:28.228241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8008498-8ae8-4a5a-92d9-a6b45fcbe110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:28.229732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8008498-8ae8-4a5a-92d9-a6b45fcbe110 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:28.230297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8008498-8ae8-4a5a-92d9-a6b45fcbe110 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:28.230791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8008498-8ae8-4a5a-92d9-a6b45fcbe110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:28.232399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ca8633ac-57b8-4a3f-b670-4819f89fe8e9 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:28.233687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ca8633ac-57b8-4a3f-b670-4819f89fe8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:28.249916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8008498-8ae8-4a5a-92d9-a6b45fcbe110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:28.252241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8008498-8ae8-4a5a-92d9-a6b45fcbe110 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-b8008498-8ae8-4a5a-92d9-a6b45fcbe110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:28.278549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:28.279522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:28.280400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:28.280400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:28.280400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:28.280400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:28.280400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 333/636] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:28.281635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8008498-8ae8-4a5a-92d9-a6b45fcbe110 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:28.303085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:28.339743 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 304/637] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:28.341141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8633ac-57b8-4a3f-b670-4819f89fe8e9 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:28.437168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8008498-8ae8-4a5a-92d9-a6b45fcbe110 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:28.489563 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:28.500744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8633ac-57b8-4a3f-b670-4819f89fe8e9 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:28.504264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4fe7c-7e27-4974-ad7f-b009b42a54c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:28.505083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4fe7c-7e27-4974-ad7f-b009b42a54c8 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:28.505392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4fe7c-7e27-4974-ad7f-b009b42a54c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:28.505663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4fe7c-7e27-4974-ad7f-b009b42a54c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:28.516982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4fe7c-7e27-4974-ad7f-b009b42a54c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-b0a4fe7c-7e27-4974-ad7f-b009b42a54c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:28.525140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:28.526329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:28.527328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 334/638] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:28.528171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4fe7c-7e27-4974-ad7f-b009b42a54c8 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:28.551293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ad12e-9eb0-4944-a3f4-fe433d9f8633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:28.551965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ad12e-9eb0-4944-a3f4-fe433d9f8633 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:28.552468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ad12e-9eb0-4944-a3f4-fe433d9f8633 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:28.552600 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ad12e-9eb0-4944-a3f4-fe433d9f8633 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:28.553257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ad12e-9eb0-4944-a3f4-fe433d9f8633 None admin] RBAC: Adding query filter params (name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:28.553598 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ad12e-9eb0-4944-a3f4-fe433d9f8633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:28.555100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ad12e-9eb0-4944-a3f4-fe433d9f8633 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:28.565203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 305/639] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:28.565885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ad12e-9eb0-4944-a3f4-fe433d9f8633 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:28.634477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:28.663623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4fe7c-7e27-4974-ad7f-b009b42a54c8 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:28.699807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ad12e-9eb0-4944-a3f4-fe433d9f8633 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:28.707853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866107af-6eb9-4d13-8e3f-93e0c149d0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:28.709242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866107af-6eb9-4d13-8e3f-93e0c149d0d2 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:28.709603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866107af-6eb9-4d13-8e3f-93e0c149d0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:28.709845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866107af-6eb9-4d13-8e3f-93e0c149d0d2 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:28.710615 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866107af-6eb9-4d13-8e3f-93e0c149d0d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:28.711440 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866107af-6eb9-4d13-8e3f-93e0c149d0d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:28.713477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866107af-6eb9-4d13-8e3f-93e0c149d0d2 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:28.730676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 335/640] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:28.732161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866107af-6eb9-4d13-8e3f-93e0c149d0d2 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:28.752317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9896b1f-ab9c-4c95-ac7c-691257b71635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:28.753179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9896b1f-ab9c-4c95-ac7c-691257b71635 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:28.753480 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9896b1f-ab9c-4c95-ac7c-691257b71635 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:28.753972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9896b1f-ab9c-4c95-ac7c-691257b71635 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:28.768939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9896b1f-ab9c-4c95-ac7c-691257b71635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-c9896b1f-ab9c-4c95-ac7c-691257b71635 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:28.777560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:28.778513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:28.779333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 306/641] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:28.780014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9896b1f-ab9c-4c95-ac7c-691257b71635 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:28.809208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:28.864675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866107af-6eb9-4d13-8e3f-93e0c149d0d2 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:28.909950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8e90e-d1d8-4cc4-a051-4cd35ca389b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:28.910642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8e90e-d1d8-4cc4-a051-4cd35ca389b8 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:28.910920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8e90e-d1d8-4cc4-a051-4cd35ca389b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:28.911202 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8e90e-d1d8-4cc4-a051-4cd35ca389b8 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:28.911959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8e90e-d1d8-4cc4-a051-4cd35ca389b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:28.912271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8e90e-d1d8-4cc4-a051-4cd35ca389b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:28.913658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8e90e-d1d8-4cc4-a051-4cd35ca389b8 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:28.915216 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:28.918760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9896b1f-ab9c-4c95-ac7c-691257b71635 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:28.926869 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 336/642] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:28.930409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8e90e-d1d8-4cc4-a051-4cd35ca389b8 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:28.958520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12758288-4ec8-46eb-a567-f7d6cb8615aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:28.959240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12758288-4ec8-46eb-a567-f7d6cb8615aa None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:28.959525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12758288-4ec8-46eb-a567-f7d6cb8615aa None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:28.959808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12758288-4ec8-46eb-a567-f7d6cb8615aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:28.972339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12758288-4ec8-46eb-a567-f7d6cb8615aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:28.973848 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12758288-4ec8-46eb-a567-f7d6cb8615aa None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-12758288-4ec8-46eb-a567-f7d6cb8615aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:28.993053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:28.994931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:28.996408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:28.996408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:28.996408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:28.996408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:28.996408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 307/643] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:28.996408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12758288-4ec8-46eb-a567-f7d6cb8615aa None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:29.066863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8e90e-d1d8-4cc4-a051-4cd35ca389b8 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:29.083702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:29.119363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a6959-48a9-4a38-9234-42bd6a67b6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:29.120166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a6959-48a9-4a38-9234-42bd6a67b6ae None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:29.120451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a6959-48a9-4a38-9234-42bd6a67b6ae None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:29.120752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a6959-48a9-4a38-9234-42bd6a67b6ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:29.122020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12758288-4ec8-46eb-a567-f7d6cb8615aa None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:29.128580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a6959-48a9-4a38-9234-42bd6a67b6ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:29.130187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a6959-48a9-4a38-9234-42bd6a67b6ae None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-4d4a6959-48a9-4a38-9234-42bd6a67b6ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:29.145521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:29.146646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:29.147980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:29.147980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:29.147980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:29.147980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:29.147980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:29.147980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:29.147980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:29.147980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:29.147980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 337/644] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:29.147980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a6959-48a9-4a38-9234-42bd6a67b6ae None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:29.165693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48230c-1534-4f81-9009-499b004aa402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:29.166379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48230c-1534-4f81-9009-499b004aa402 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:29.166667 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48230c-1534-4f81-9009-499b004aa402 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:29.166949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48230c-1534-4f81-9009-499b004aa402 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:29.167775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48230c-1534-4f81-9009-499b004aa402 None admin] RBAC: Adding query filter params (name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:29.168087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48230c-1534-4f81-9009-499b004aa402 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:29.169525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48230c-1534-4f81-9009-499b004aa402 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:29.178530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 308/645] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:29.179388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48230c-1534-4f81-9009-499b004aa402 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:29.219100 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:29.276257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a6959-48a9-4a38-9234-42bd6a67b6ae None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:29.290792 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48230c-1534-4f81-9009-499b004aa402 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:29.319858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c93a8-7d1f-4d6a-b9ab-21ab7965152b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:29.320780 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c93a8-7d1f-4d6a-b9ab-21ab7965152b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:29.321065 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c93a8-7d1f-4d6a-b9ab-21ab7965152b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:29.321379 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c93a8-7d1f-4d6a-b9ab-21ab7965152b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:29.321705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:29.328594 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71331d8c-f947-4371-b72d-629d95388693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:29.329426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71331d8c-f947-4371-b72d-629d95388693 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:29.329824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71331d8c-f947-4371-b72d-629d95388693 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:29.330203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71331d8c-f947-4371-b72d-629d95388693 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:29.331059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71331d8c-f947-4371-b72d-629d95388693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:29.331476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71331d8c-f947-4371-b72d-629d95388693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:29.333121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71331d8c-f947-4371-b72d-629d95388693 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:29.338529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c93a8-7d1f-4d6a-b9ab-21ab7965152b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:29.341421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c93a8-7d1f-4d6a-b9ab-21ab7965152b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:29.354759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 309/646] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:29.354759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71331d8c-f947-4371-b72d-629d95388693 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-605c93a8-7d1f-4d6a-b9ab-21ab7965152b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:29.376028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:29.377133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:29.378639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:29.378639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:29.378639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:29.378639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:29.385691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 338/647] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:29.387792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c93a8-7d1f-4d6a-b9ab-21ab7965152b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:29.493693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71331d8c-f947-4371-b72d-629d95388693 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:29.538462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7685528-23d3-4f47-a127-e9a8397321d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:29.539338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7685528-23d3-4f47-a127-e9a8397321d9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:29.539758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7685528-23d3-4f47-a127-e9a8397321d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:29.540134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7685528-23d3-4f47-a127-e9a8397321d9 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:29.541050 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7685528-23d3-4f47-a127-e9a8397321d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:29.541526 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7685528-23d3-4f47-a127-e9a8397321d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:29.541903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c93a8-7d1f-4d6a-b9ab-21ab7965152b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:29.543239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7685528-23d3-4f47-a127-e9a8397321d9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:29.557690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:29.560772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 310/648] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:29.562101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7685528-23d3-4f47-a127-e9a8397321d9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:29.596735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4280b4-ddea-418a-b509-6b13b8f27082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:29.597685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4280b4-ddea-418a-b509-6b13b8f27082 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:29.597888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4280b4-ddea-418a-b509-6b13b8f27082 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:29.598124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4280b4-ddea-418a-b509-6b13b8f27082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:29.609849 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4280b4-ddea-418a-b509-6b13b8f27082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:29.612383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4280b4-ddea-418a-b509-6b13b8f27082 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:29.626518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-8f4280b4-ddea-418a-b509-6b13b8f27082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:29.632488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:29.633592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:29.639873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:29.639873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:29.639873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:29.639873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:29.639873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 339/649] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:29.639873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4280b4-ddea-418a-b509-6b13b8f27082 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:29.696145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7685528-23d3-4f47-a127-e9a8397321d9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:29.742998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440473d1-86c4-4b1b-803f-f72eb77b1bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:29.743919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440473d1-86c4-4b1b-803f-f72eb77b1bf0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:29.744182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440473d1-86c4-4b1b-803f-f72eb77b1bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:29.744398 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440473d1-86c4-4b1b-803f-f72eb77b1bf0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:29.755832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440473d1-86c4-4b1b-803f-f72eb77b1bf0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:29.757478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440473d1-86c4-4b1b-803f-f72eb77b1bf0 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:29.773458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4280b4-ddea-418a-b509-6b13b8f27082 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-440473d1-86c4-4b1b-803f-f72eb77b1bf0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:29.774161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:29.776048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 311/650] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:29.778747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440473d1-86c4-4b1b-803f-f72eb77b1bf0 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:29.828432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc305e23-be38-49a3-ad15-2700f9e99e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:29.829128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc305e23-be38-49a3-ad15-2700f9e99e57 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:29.829474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc305e23-be38-49a3-ad15-2700f9e99e57 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:29.829757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc305e23-be38-49a3-ad15-2700f9e99e57 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:29.830561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc305e23-be38-49a3-ad15-2700f9e99e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:29.830833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc305e23-be38-49a3-ad15-2700f9e99e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:29.832288 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc305e23-be38-49a3-ad15-2700f9e99e57 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:29.846951 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 340/651] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:29.852600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc305e23-be38-49a3-ad15-2700f9e99e57 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:29.887064 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:29.931729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440473d1-86c4-4b1b-803f-f72eb77b1bf0 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:29.947389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:29.975826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3e862-c524-4a3f-a1b7-8de52c7ec7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:29.976503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e862-c524-4a3f-a1b7-8de52c7ec7e8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:29.976783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e862-c524-4a3f-a1b7-8de52c7ec7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:29.977054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3e862-c524-4a3f-a1b7-8de52c7ec7e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:29.977936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3e862-c524-4a3f-a1b7-8de52c7ec7e8 None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=e1d09e168fc54f1da06b719989ae8ef0) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:29.978245 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3e862-c524-4a3f-a1b7-8de52c7ec7e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:29.979725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3e862-c524-4a3f-a1b7-8de52c7ec7e8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:29.987223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 312/652] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:29 2022] GET /identity/v3/role_assignments?user.id=e1d09e168fc54f1da06b719989ae8ef0&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:29.988002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3e862-c524-4a3f-a1b7-8de52c7ec7e8 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:29.998944 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc305e23-be38-49a3-ad15-2700f9e99e57 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:30.053560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69defbed-31d1-4f06-8d52-1efc04e966a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:30.054711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69defbed-31d1-4f06-8d52-1efc04e966a2 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:30.055183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69defbed-31d1-4f06-8d52-1efc04e966a2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:30.055651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69defbed-31d1-4f06-8d52-1efc04e966a2 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:30.056980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69defbed-31d1-4f06-8d52-1efc04e966a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:30.057529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69defbed-31d1-4f06-8d52-1efc04e966a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:30.060073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69defbed-31d1-4f06-8d52-1efc04e966a2 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:30.081568 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 341/653] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:30.081568 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69defbed-31d1-4f06-8d52-1efc04e966a2 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:30.155782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3e862-c524-4a3f-a1b7-8de52c7ec7e8 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:30.198130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:30.215150 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd10096-2f22-4880-b119-dbe6059457e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:30.215929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd10096-2f22-4880-b119-dbe6059457e9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:30.216204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd10096-2f22-4880-b119-dbe6059457e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:30.216483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd10096-2f22-4880-b119-dbe6059457e9 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:30.217341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd10096-2f22-4880-b119-dbe6059457e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:30.217729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd10096-2f22-4880-b119-dbe6059457e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:30.219211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd10096-2f22-4880-b119-dbe6059457e9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:30.226697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69defbed-31d1-4f06-8d52-1efc04e966a2 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:30.244006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 313/654] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jul 19 20:35:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:30.244935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd10096-2f22-4880-b119-dbe6059457e9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:30.271936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5465d2f-607b-472f-bb22-3f3c191fc529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:30.272724 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5465d2f-607b-472f-bb22-3f3c191fc529 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:30.273017 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5465d2f-607b-472f-bb22-3f3c191fc529 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:30.273313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5465d2f-607b-472f-bb22-3f3c191fc529 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:30.282738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5465d2f-607b-472f-bb22-3f3c191fc529 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:30.284219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5465d2f-607b-472f-bb22-3f3c191fc529 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:30.291693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-c5465d2f-607b-472f-bb22-3f3c191fc529 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:30.302274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:30.303446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:30.304544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:30.304544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:30.304544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:30.304544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:30.304544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:30.304544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:30.304544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:30.304544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:30.304544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:35:30.304544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:30.306118 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 342/655] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:30.307369 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd2b3d-dd99-4e67-ba3f-be63c7d73586 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:30.307658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd2b3d-dd99-4e67-ba3f-be63c7d73586 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:30.307937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd2b3d-dd99-4e67-ba3f-be63c7d73586 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:30.309391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 343/656] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:30.315058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd2b3d-dd99-4e67-ba3f-be63c7d73586 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:30.400743 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd10096-2f22-4880-b119-dbe6059457e9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:30.443011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8f63f-e8b5-4ac0-ad16-f3ae1905eca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:30.444138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8f63f-e8b5-4ac0-ad16-f3ae1905eca4 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:30.444552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8f63f-e8b5-4ac0-ad16-f3ae1905eca4 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:30.444897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8f63f-e8b5-4ac0-ad16-f3ae1905eca4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:30.453784 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8f63f-e8b5-4ac0-ad16-f3ae1905eca4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:30.455544 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8f63f-e8b5-4ac0-ad16-f3ae1905eca4 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:30.465114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd2b3d-dd99-4e67-ba3f-be63c7d73586 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-09b8f63f-e8b5-4ac0-ad16-f3ae1905eca4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:30.473274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:30.474616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:30.475740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:30.475740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:30.475740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:30.475740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:30.475740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:30.475740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:30.475740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:30.475740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:30.475740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:35:30.475740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:30.476900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 314/657] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:30.477936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8f63f-e8b5-4ac0-ad16-f3ae1905eca4 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:30.518536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b624f-b3d6-43e3-ac56-017f435ab1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:30.519236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b624f-b3d6-43e3-ac56-017f435ab1f2 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:30.519525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b624f-b3d6-43e3-ac56-017f435ab1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:30.519806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b624f-b3d6-43e3-ac56-017f435ab1f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:30.535087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:30.616951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:30.625638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8f63f-e8b5-4ac0-ad16-f3ae1905eca4 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:30.632900 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b624f-b3d6-43e3-ac56-017f435ab1f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:30.634501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b624f-b3d6-43e3-ac56-017f435ab1f2 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:30.675451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab4d62-c5b2-4dfe-afc5-ef84c2469987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:30.676132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab4d62-c5b2-4dfe-afc5-ef84c2469987 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:30.676414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab4d62-c5b2-4dfe-afc5-ef84c2469987 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:30.676689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab4d62-c5b2-4dfe-afc5-ef84c2469987 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:30.677573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab4d62-c5b2-4dfe-afc5-ef84c2469987 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:30.677922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab4d62-c5b2-4dfe-afc5-ef84c2469987 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:30.679452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab4d62-c5b2-4dfe-afc5-ef84c2469987 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:30.703371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 315/658] 10.222.0.70 () {56 vars in 1137 bytes} [Tue Jul 19 20:35:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:30.705152 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f41c29-381b-4436-a0a3-f11f67427a0a None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:30.705511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f41c29-381b-4436-a0a3-f11f67427a0a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:30.705842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f41c29-381b-4436-a0a3-f11f67427a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:30.727750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-96f41c29-381b-4436-a0a3-f11f67427a0a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:30.729865 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-96f41c29-381b-4436-a0a3-f11f67427a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:30.789701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 344/659] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:30.789701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b624f-b3d6-43e3-ac56-017f435ab1f2 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:30.840028 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 316/660] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:30.841215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f41c29-381b-4436-a0a3-f11f67427a0a None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:30.849038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:30.941519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b624f-b3d6-43e3-ac56-017f435ab1f2 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:30.968481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:30.997926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd01cb-6eea-4762-be70-9a40f2d6f98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:30.998735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd01cb-6eea-4762-be70-9a40f2d6f98f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:30.999151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd01cb-6eea-4762-be70-9a40f2d6f98f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:31.000209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd01cb-6eea-4762-be70-9a40f2d6f98f None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:31.000429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd01cb-6eea-4762-be70-9a40f2d6f98f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:31.000647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd01cb-6eea-4762-be70-9a40f2d6f98f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:31.002903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd01cb-6eea-4762-be70-9a40f2d6f98f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:31.003889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f41c29-381b-4436-a0a3-f11f67427a0a None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:31.037152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 345/661] 10.222.0.70 () {56 vars in 1139 bytes} [Tue Jul 19 20:35:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:31.038984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd01cb-6eea-4762-be70-9a40f2d6f98f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:31.061050 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd95f9-588a-4571-9cf1-3c5c36908f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:31.061975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd95f9-588a-4571-9cf1-3c5c36908f3c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:31.062438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd95f9-588a-4571-9cf1-3c5c36908f3c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:31.062913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd95f9-588a-4571-9cf1-3c5c36908f3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:31.173041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd95f9-588a-4571-9cf1-3c5c36908f3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:31.175883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd95f9-588a-4571-9cf1-3c5c36908f3c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:31.183139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:31.187592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd01cb-6eea-4762-be70-9a40f2d6f98f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:31.237408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b013caa-fc3f-46f9-bf10-d2352d3106a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:31.238244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b013caa-fc3f-46f9-bf10-d2352d3106a3 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:31.238522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b013caa-fc3f-46f9-bf10-d2352d3106a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:31.238816 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b013caa-fc3f-46f9-bf10-d2352d3106a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:31.250837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b013caa-fc3f-46f9-bf10-d2352d3106a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:31.252375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b013caa-fc3f-46f9-bf10-d2352d3106a3 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-5b013caa-fc3f-46f9-bf10-d2352d3106a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:31.272778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:31.274517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:31.275534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:31.275534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:31.275534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:31.275534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:31.279622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 346/662] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:31.280615 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b013caa-fc3f-46f9-bf10-d2352d3106a3 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:31.332249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:31.361058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 317/663] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:31.362000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd95f9-588a-4571-9cf1-3c5c36908f3c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:31.452855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b013caa-fc3f-46f9-bf10-d2352d3106a3 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:31.513246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09080044-4f6d-4547-ad85-efcdcf1d5704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:31.514310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09080044-4f6d-4547-ad85-efcdcf1d5704 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:31.514704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09080044-4f6d-4547-ad85-efcdcf1d5704 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:31.515177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09080044-4f6d-4547-ad85-efcdcf1d5704 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:31.520099 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd95f9-588a-4571-9cf1-3c5c36908f3c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:31.528184 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09080044-4f6d-4547-ad85-efcdcf1d5704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-09080044-4f6d-4547-ad85-efcdcf1d5704 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:31.536181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:31.537232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 347/664] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:31.538797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09080044-4f6d-4547-ad85-efcdcf1d5704 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:31.558377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:31.580332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b9f6c-af6f-416e-a309-35f68d2b4aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:31.581350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b9f6c-af6f-416e-a309-35f68d2b4aec None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:31.581818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b9f6c-af6f-416e-a309-35f68d2b4aec None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:31.582306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b9f6c-af6f-416e-a309-35f68d2b4aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:31.653925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:31.671346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09080044-4f6d-4547-ad85-efcdcf1d5704 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:31.677350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b9f6c-af6f-416e-a309-35f68d2b4aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:31.678858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b9f6c-af6f-416e-a309-35f68d2b4aec None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:31.726526 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdea987-75ff-48fb-9fe2-f14c2060e44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:31.727254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdea987-75ff-48fb-9fe2-f14c2060e44d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:31.727546 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdea987-75ff-48fb-9fe2-f14c2060e44d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:31.727986 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdea987-75ff-48fb-9fe2-f14c2060e44d None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:31.729095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdea987-75ff-48fb-9fe2-f14c2060e44d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:31.729458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdea987-75ff-48fb-9fe2-f14c2060e44d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:31.731050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdea987-75ff-48fb-9fe2-f14c2060e44d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:31.750080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 348/665] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:31.751928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdea987-75ff-48fb-9fe2-f14c2060e44d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:31.814191 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 318/666] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:31.815304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b9f6c-af6f-416e-a309-35f68d2b4aec None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:31.853571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:31.916968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdea987-75ff-48fb-9fe2-f14c2060e44d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:31.929404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:31.972879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b9f6c-af6f-416e-a309-35f68d2b4aec None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:31.978919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4444b8-0789-4e93-b87c-51175902664f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:31.980723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4444b8-0789-4e93-b87c-51175902664f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:31.981515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4444b8-0789-4e93-b87c-51175902664f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:31.982245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4444b8-0789-4e93-b87c-51175902664f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:31.997005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4444b8-0789-4e93-b87c-51175902664f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:31.998640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4444b8-0789-4e93-b87c-51175902664f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-0a4444b8-0789-4e93-b87c-51175902664f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:32.020143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:32.021818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:32.022707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:32.022707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:32.022707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:32.022707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:32.022932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d85a9b-c4e1-44bd-860b-44ad249a16bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:32.022932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d85a9b-c4e1-44bd-860b-44ad249a16bf None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.022932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d85a9b-c4e1-44bd-860b-44ad249a16bf None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.022932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d85a9b-c4e1-44bd-860b-44ad249a16bf None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.023184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d85a9b-c4e1-44bd-860b-44ad249a16bf None admin] RBAC: Adding query filter params (name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:32.023184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d85a9b-c4e1-44bd-860b-44ad249a16bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:32.023983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d85a9b-c4e1-44bd-860b-44ad249a16bf None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:32.027402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 349/667] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:32.027909 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4444b8-0789-4e93-b87c-51175902664f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:32.039928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 319/668] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:32.040513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d85a9b-c4e1-44bd-860b-44ad249a16bf None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:32.156451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d85a9b-c4e1-44bd-860b-44ad249a16bf None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:32.166397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4444b8-0789-4e93-b87c-51175902664f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:32.193561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d824b-3781-4690-b335-3c97f481ce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:32.194384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d824b-3781-4690-b335-3c97f481ce1c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.194671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d824b-3781-4690-b335-3c97f481ce1c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.194949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d824b-3781-4690-b335-3c97f481ce1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.197299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:32.208370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d824b-3781-4690-b335-3c97f481ce1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:32.210130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d824b-3781-4690-b335-3c97f481ce1c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:32.211009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbe75f-6a64-4120-9e08-0838bb27a0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:32.211930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbe75f-6a64-4120-9e08-0838bb27a0b2 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.212235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbe75f-6a64-4120-9e08-0838bb27a0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.212496 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbe75f-6a64-4120-9e08-0838bb27a0b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.213120 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:32.224284 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dbe75f-6a64-4120-9e08-0838bb27a0b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:32.225780 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dbe75f-6a64-4120-9e08-0838bb27a0b2 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-5a5d824b-3781-4690-b335-3c97f481ce1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:32.232163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:32.239399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:32.240546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:32.240546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:32.240546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:32.240546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:32.240546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 320/669] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:32.240546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d824b-3781-4690-b335-3c97f481ce1c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-41dbe75f-6a64-4120-9e08-0838bb27a0b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:32.251220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:32.252350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:32.253245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:32.253245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:32.253245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:32.253245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:32.253245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:32.253245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:32.253245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:32.253245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:32.253245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 350/670] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:32.253779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbe75f-6a64-4120-9e08-0838bb27a0b2 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:32.374123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d824b-3781-4690-b335-3c97f481ce1c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:32.387883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbe75f-6a64-4120-9e08-0838bb27a0b2 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:32.408760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f35021-5ed9-4483-9a7d-e987419d08dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:32.409345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f35021-5ed9-4483-9a7d-e987419d08dd None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.409762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f35021-5ed9-4483-9a7d-e987419d08dd None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.410029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f35021-5ed9-4483-9a7d-e987419d08dd None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.410680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f35021-5ed9-4483-9a7d-e987419d08dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:32.410987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f35021-5ed9-4483-9a7d-e987419d08dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:32.412346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f35021-5ed9-4483-9a7d-e987419d08dd None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:32.423104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468f2f7-af10-4634-a61b-58055337ebf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:32.423899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468f2f7-af10-4634-a61b-58055337ebf7 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.424187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468f2f7-af10-4634-a61b-58055337ebf7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.424468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468f2f7-af10-4634-a61b-58055337ebf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.424605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 321/671] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:32.429077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f35021-5ed9-4483-9a7d-e987419d08dd None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:32.436114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468f2f7-af10-4634-a61b-58055337ebf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:32.437600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468f2f7-af10-4634-a61b-58055337ebf7 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-2468f2f7-af10-4634-a61b-58055337ebf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:32.455364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:32.457130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:32.459377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:32.459377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:32.459377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:32.459377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:32.459377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 351/672] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:32.459377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468f2f7-af10-4634-a61b-58055337ebf7 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:32.489165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:32.501551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:32.539690 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f35021-5ed9-4483-9a7d-e987419d08dd None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:32.576871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83438f84-a404-49e5-b178-a885c42949fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:32.577780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83438f84-a404-49e5-b178-a885c42949fe None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.578164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83438f84-a404-49e5-b178-a885c42949fe None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.578635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83438f84-a404-49e5-b178-a885c42949fe None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.579542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83438f84-a404-49e5-b178-a885c42949fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:32.579932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83438f84-a404-49e5-b178-a885c42949fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:32.581522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83438f84-a404-49e5-b178-a885c42949fe None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:32.585217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468f2f7-af10-4634-a61b-58055337ebf7 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:32.596707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 322/673] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:32.598992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de142b2c-4e3d-44db-952c-b169df945721 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.599523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de142b2c-4e3d-44db-952c-b169df945721 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.599972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de142b2c-4e3d-44db-952c-b169df945721 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.602557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 323/674] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:32.602557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de142b2c-4e3d-44db-952c-b169df945721 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:32.644107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6485-0b36-4f5f-b03f-ce290ab5cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:32.645243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6485-0b36-4f5f-b03f-ce290ab5cc6a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.646325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6485-0b36-4f5f-b03f-ce290ab5cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.646325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6485-0b36-4f5f-b03f-ce290ab5cc6a None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.647776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d6485-0b36-4f5f-b03f-ce290ab5cc6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:32.648337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d6485-0b36-4f5f-b03f-ce290ab5cc6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:32.651086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d6485-0b36-4f5f-b03f-ce290ab5cc6a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:32.678836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 352/675] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:32.678836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6485-0b36-4f5f-b03f-ce290ab5cc6a None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:32.721965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de142b2c-4e3d-44db-952c-b169df945721 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:32.741454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:32.771086 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514653f-8f9e-4d0d-8fb9-ee0349692ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:32.771587 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514653f-8f9e-4d0d-8fb9-ee0349692ac5 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.771875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514653f-8f9e-4d0d-8fb9-ee0349692ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.772152 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514653f-8f9e-4d0d-8fb9-ee0349692ac5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.782788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514653f-8f9e-4d0d-8fb9-ee0349692ac5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:32.784468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514653f-8f9e-4d0d-8fb9-ee0349692ac5 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-3514653f-8f9e-4d0d-8fb9-ee0349692ac5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:32.799999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:32.801849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:32.803251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:32.803251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:32.803251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:32.803251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:32.803251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:32.803251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:32.803251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:32.803251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:32.803251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 324/676] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:32.803251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514653f-8f9e-4d0d-8fb9-ee0349692ac5 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:32.836550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:32.845788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6485-0b36-4f5f-b03f-ce290ab5cc6a None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:32.888512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716c97d-61ad-4db2-b2ae-dee8c3c9394d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:32.889629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716c97d-61ad-4db2-b2ae-dee8c3c9394d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.890102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716c97d-61ad-4db2-b2ae-dee8c3c9394d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.890572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716c97d-61ad-4db2-b2ae-dee8c3c9394d None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.891860 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716c97d-61ad-4db2-b2ae-dee8c3c9394d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:32.892399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716c97d-61ad-4db2-b2ae-dee8c3c9394d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:32.894916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716c97d-61ad-4db2-b2ae-dee8c3c9394d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:32.914519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 353/677] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:32.915862 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993d570-3b49-4dc0-8760-649e55fdd46b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.916134 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993d570-3b49-4dc0-8760-649e55fdd46b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.916413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993d570-3b49-4dc0-8760-649e55fdd46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.924703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514653f-8f9e-4d0d-8fb9-ee0349692ac5 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:32.933555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a993d570-3b49-4dc0-8760-649e55fdd46b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:32.934684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a993d570-3b49-4dc0-8760-649e55fdd46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:32.972602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac2bfe-d0d8-4183-ab16-a9d40b2d6242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:32.973734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac2bfe-d0d8-4183-ab16-a9d40b2d6242 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:32.974108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac2bfe-d0d8-4183-ab16-a9d40b2d6242 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:32.974457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac2bfe-d0d8-4183-ab16-a9d40b2d6242 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:32.982253 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac2bfe-d0d8-4183-ab16-a9d40b2d6242 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:32.983797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac2bfe-d0d8-4183-ab16-a9d40b2d6242 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-97ac2bfe-d0d8-4183-ab16-a9d40b2d6242 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:32.999049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:33.000346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:33.001782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:33.001782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:33.001782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:33.001782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:33.001782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:33.001782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:33.001782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:33.001782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:33.001782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 325/678] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:33.001782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac2bfe-d0d8-4183-ab16-a9d40b2d6242 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:33.018178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 354/679] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:33.019062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993d570-3b49-4dc0-8760-649e55fdd46b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:33.043096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:33.114127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac2bfe-d0d8-4183-ab16-a9d40b2d6242 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:33.134731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:33.156654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993d570-3b49-4dc0-8760-649e55fdd46b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:33.160926 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c402b-b8b3-48b0-b9ed-39ce06717f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:33.161624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c402b-b8b3-48b0-b9ed-39ce06717f73 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:33.161906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c402b-b8b3-48b0-b9ed-39ce06717f73 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:33.162178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c402b-b8b3-48b0-b9ed-39ce06717f73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:33.163045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c402b-b8b3-48b0-b9ed-39ce06717f73 None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=82e1192b2d554199a08debb4aa392479) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:33.163359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c402b-b8b3-48b0-b9ed-39ce06717f73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:33.164807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c402b-b8b3-48b0-b9ed-39ce06717f73 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:33.174628 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 326/680] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:33 2022] GET /identity/v3/role_assignments?user.id=82e1192b2d554199a08debb4aa392479&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:33.175200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c402b-b8b3-48b0-b9ed-39ce06717f73 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:33.206031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012173b3-91d4-4ea6-ba01-f3c51ccfbbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:33.206643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012173b3-91d4-4ea6-ba01-f3c51ccfbbfb None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:33.206935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012173b3-91d4-4ea6-ba01-f3c51ccfbbfb None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:33.207216 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012173b3-91d4-4ea6-ba01-f3c51ccfbbfb None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:33.208067 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012173b3-91d4-4ea6-ba01-f3c51ccfbbfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:33.208406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012173b3-91d4-4ea6-ba01-f3c51ccfbbfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:33.210097 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012173b3-91d4-4ea6-ba01-f3c51ccfbbfb None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:33.223554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 355/681] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:33.224431 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012173b3-91d4-4ea6-ba01-f3c51ccfbbfb None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:33.311935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:33.315572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c402b-b8b3-48b0-b9ed-39ce06717f73 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:33.361909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8e8ee-915c-4cd9-b211-5bd0262b7924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:33.363158 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8e8ee-915c-4cd9-b211-5bd0262b7924 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:33.363588 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8e8ee-915c-4cd9-b211-5bd0262b7924 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:33.364430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8e8ee-915c-4cd9-b211-5bd0262b7924 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:33.375443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8e8ee-915c-4cd9-b211-5bd0262b7924 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:33.377632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8e8ee-915c-4cd9-b211-5bd0262b7924 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:33.389568 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012173b3-91d4-4ea6-ba01-f3c51ccfbbfb None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-96d8e8ee-915c-4cd9-b211-5bd0262b7924 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:33.394953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:33.396063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:33.396989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:33.396989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:33.396989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:33.396989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:33.396989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:33.396989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:33.396989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:33.396989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:33.396989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:33.396989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:33.401100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 327/682] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:33.403431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8e8ee-915c-4cd9-b211-5bd0262b7924 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:33.436655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2a732-7a56-4079-b0b6-11f6246ccce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:33.437558 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a732-7a56-4079-b0b6-11f6246ccce8 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:33.437994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a732-7a56-4079-b0b6-11f6246ccce8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:33.438314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a732-7a56-4079-b0b6-11f6246ccce8 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:33.439141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2a732-7a56-4079-b0b6-11f6246ccce8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:33.439480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2a732-7a56-4079-b0b6-11f6246ccce8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:33.442027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2a732-7a56-4079-b0b6-11f6246ccce8 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:33.465127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 356/683] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:33.466596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2a732-7a56-4079-b0b6-11f6246ccce8 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:33.474324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:33.555085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8e8ee-915c-4cd9-b211-5bd0262b7924 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:33.595040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1c556-6bf1-42e0-8276-178438fcacae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:33.595582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2a732-7a56-4079-b0b6-11f6246ccce8 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:33.595838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1c556-6bf1-42e0-8276-178438fcacae None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:33.596117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1c556-6bf1-42e0-8276-178438fcacae None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:33.596524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1c556-6bf1-42e0-8276-178438fcacae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:33.608785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1c556-6bf1-42e0-8276-178438fcacae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-60d1c556-6bf1-42e0-8276-178438fcacae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:33.617106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:33.618121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:33.618950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 328/684] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:33.620226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1c556-6bf1-42e0-8276-178438fcacae None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:33.636425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff3618-83d4-424c-aa85-0920c020fb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:33.637117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff3618-83d4-424c-aa85-0920c020fb71 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:33.637438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff3618-83d4-424c-aa85-0920c020fb71 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:33.637737 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff3618-83d4-424c-aa85-0920c020fb71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:33.638611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff3618-83d4-424c-aa85-0920c020fb71 None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=a75951e9512c420992a4fd6c1a6b1176) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:33.638919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff3618-83d4-424c-aa85-0920c020fb71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:33.640356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff3618-83d4-424c-aa85-0920c020fb71 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:33.644850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:33.647842 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 357/685] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:33 2022] GET /identity/v3/role_assignments?user.id=a75951e9512c420992a4fd6c1a6b1176&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:33.648589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff3618-83d4-424c-aa85-0920c020fb71 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:33.736132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:33.779965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff3618-83d4-424c-aa85-0920c020fb71 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:33.782406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1c556-6bf1-42e0-8276-178438fcacae None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:33.830522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c1c29-4597-4f62-9b27-9c3c5d0e1224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:33.831604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c1c29-4597-4f62-9b27-9c3c5d0e1224 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:33.832159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c1c29-4597-4f62-9b27-9c3c5d0e1224 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:33.832658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c1c29-4597-4f62-9b27-9c3c5d0e1224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:33.833928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c1c29-4597-4f62-9b27-9c3c5d0e1224 None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=fb186ee2e3c94ca7ad203c9c0e32e498) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:33.834491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c1c29-4597-4f62-9b27-9c3c5d0e1224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:33.836345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c1c29-4597-4f62-9b27-9c3c5d0e1224 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:33.842251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec9279-2ab0-4a19-835b-dd9513e281e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:33.843520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec9279-2ab0-4a19-835b-dd9513e281e9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:33.844467 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec9279-2ab0-4a19-835b-dd9513e281e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:33.845331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec9279-2ab0-4a19-835b-dd9513e281e9 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:33.845579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 358/686] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:33 2022] GET /identity/v3/role_assignments?user.id=fb186ee2e3c94ca7ad203c9c0e32e498&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:33.846502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec9279-2ab0-4a19-835b-dd9513e281e9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:33.846910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec9279-2ab0-4a19-835b-dd9513e281e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:33.847249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c1c29-4597-4f62-9b27-9c3c5d0e1224 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:33.849529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec9279-2ab0-4a19-835b-dd9513e281e9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:33.887714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 329/687] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jul 19 20:35:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:33.892879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec9279-2ab0-4a19-835b-dd9513e281e9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:34.017624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c1c29-4597-4f62-9b27-9c3c5d0e1224 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:34.085605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ea8e5-a75d-41ed-831b-11807a0d9e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:34.086992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ea8e5-a75d-41ed-831b-11807a0d9e60 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:34.087285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ea8e5-a75d-41ed-831b-11807a0d9e60 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:34.087572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ea8e5-a75d-41ed-831b-11807a0d9e60 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:34.088383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ea8e5-a75d-41ed-831b-11807a0d9e60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:34.088694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ea8e5-a75d-41ed-831b-11807a0d9e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:34.091413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ea8e5-a75d-41ed-831b-11807a0d9e60 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:34.096736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:34.104170 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:34.104544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 359/688] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:34.122056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ea8e5-a75d-41ed-831b-11807a0d9e60 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:34.166577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec9279-2ab0-4a19-835b-dd9513e281e9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:34.227333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0a38a-1d8f-4384-b1bc-204505bb0d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:34.228208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0a38a-1d8f-4384-b1bc-204505bb0d28 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:34.228407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0a38a-1d8f-4384-b1bc-204505bb0d28 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:34.228634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0a38a-1d8f-4384-b1bc-204505bb0d28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:34.290051 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ea8e5-a75d-41ed-831b-11807a0d9e60 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:34.335307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0a38a-1d8f-4384-b1bc-204505bb0d28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:34.336789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0a38a-1d8f-4384-b1bc-204505bb0d28 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:34.345124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8c10f-e9c4-4b9f-b613-8fa50f235715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:34.346354 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8c10f-e9c4-4b9f-b613-8fa50f235715 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:34.346843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8c10f-e9c4-4b9f-b613-8fa50f235715 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:34.347321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8c10f-e9c4-4b9f-b613-8fa50f235715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:34.386337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8c10f-e9c4-4b9f-b613-8fa50f235715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:34.388125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8c10f-e9c4-4b9f-b613-8fa50f235715 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:34.429784 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-80e8c10f-e9c4-4b9f-b613-8fa50f235715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:34.430474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:34.431410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:34.432282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:34.432282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:34.432282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:34.432282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:34.432651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 360/689] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:34.439778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8c10f-e9c4-4b9f-b613-8fa50f235715 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:34.495016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 330/690] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:35:34.507915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0a38a-1d8f-4384-b1bc-204505bb0d28 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:34.511457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:34.601623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8c10f-e9c4-4b9f-b613-8fa50f235715 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:34.664570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7ca90-5055-4efa-b20e-fc886a6de240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:34.665921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7ca90-5055-4efa-b20e-fc886a6de240 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:34.666487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7ca90-5055-4efa-b20e-fc886a6de240 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:34.667057 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7ca90-5055-4efa-b20e-fc886a6de240 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:34.669121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7ca90-5055-4efa-b20e-fc886a6de240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:34.669834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7ca90-5055-4efa-b20e-fc886a6de240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:34.672413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7ca90-5055-4efa-b20e-fc886a6de240 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:34.692230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 361/691] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:34.702373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7ca90-5055-4efa-b20e-fc886a6de240 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:34.702704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0a38a-1d8f-4384-b1bc-204505bb0d28 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:34.776312 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67518678-b861-4c0e-88d8-e9eacaff5167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:34.778023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67518678-b861-4c0e-88d8-e9eacaff5167 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:34.778496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67518678-b861-4c0e-88d8-e9eacaff5167 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:34.778955 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67518678-b861-4c0e-88d8-e9eacaff5167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:34.799018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67518678-b861-4c0e-88d8-e9eacaff5167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:34.800779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67518678-b861-4c0e-88d8-e9eacaff5167 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-67518678-b861-4c0e-88d8-e9eacaff5167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:34.830096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:34.831248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:34.832149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:34.832149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:34.832149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:34.832149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:34.840197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 331/692] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:34.852209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67518678-b861-4c0e-88d8-e9eacaff5167 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:34.905504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7ca90-5055-4efa-b20e-fc886a6de240 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:34.912352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:34.924068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:34.976410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d5ff3-4019-424c-85aa-f63df7af3785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:34.977547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d5ff3-4019-424c-85aa-f63df7af3785 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:34.979008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d5ff3-4019-424c-85aa-f63df7af3785 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:34.980195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d5ff3-4019-424c-85aa-f63df7af3785 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:34.997755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d5ff3-4019-424c-85aa-f63df7af3785 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:35.000390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d5ff3-4019-424c-85aa-f63df7af3785 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-f37d5ff3-4019-424c-85aa-f63df7af3785 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:35.027503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:35.028761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:35.029948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:35.029948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:35.029948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:35.029948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:35.029948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:35.029948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:35.029948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:35.029948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:35.029948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:35.029948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:35.043504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 362/693] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jul 19 20:35:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:35.050591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d5ff3-4019-424c-85aa-f63df7af3785 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:35.074650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67518678-b861-4c0e-88d8-e9eacaff5167 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:35.140182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f65352-5390-4fed-bec7-fa39db2bb021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:35.141131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f65352-5390-4fed-bec7-fa39db2bb021 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:35.141615 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f65352-5390-4fed-bec7-fa39db2bb021 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:35.142043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f65352-5390-4fed-bec7-fa39db2bb021 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:35.142930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f65352-5390-4fed-bec7-fa39db2bb021 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:35.143389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f65352-5390-4fed-bec7-fa39db2bb021 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:35.145023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f65352-5390-4fed-bec7-fa39db2bb021 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:35.169098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 332/694] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:35.175618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d5ff3-4019-424c-85aa-f63df7af3785 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:35.185858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f65352-5390-4fed-bec7-fa39db2bb021 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:35.225494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d2d93-a09a-4a0c-86f5-d0f94f3898dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:35.226180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d2d93-a09a-4a0c-86f5-d0f94f3898dc None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:35.226459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d2d93-a09a-4a0c-86f5-d0f94f3898dc None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:35.226744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d2d93-a09a-4a0c-86f5-d0f94f3898dc None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:35.227590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d2d93-a09a-4a0c-86f5-d0f94f3898dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:35.227902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d2d93-a09a-4a0c-86f5-d0f94f3898dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:35.229322 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d2d93-a09a-4a0c-86f5-d0f94f3898dc None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:35.272454 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 363/695] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jul 19 20:35:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:35.275321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d2d93-a09a-4a0c-86f5-d0f94f3898dc None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:35.329781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:35.376851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:35.394441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f65352-5390-4fed-bec7-fa39db2bb021 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:35.439669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d2d93-a09a-4a0c-86f5-d0f94f3898dc None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:35.471185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30ca21-adb7-4f49-8d7b-ed519e3779e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:35.472007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30ca21-adb7-4f49-8d7b-ed519e3779e5 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:35.472293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30ca21-adb7-4f49-8d7b-ed519e3779e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:35.472569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30ca21-adb7-4f49-8d7b-ed519e3779e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:35.480483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead5b22-beba-4ba6-a81f-a0fc7dd87656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:35.481454 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead5b22-beba-4ba6-a81f-a0fc7dd87656 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:35.481948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead5b22-beba-4ba6-a81f-a0fc7dd87656 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:35.482458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead5b22-beba-4ba6-a81f-a0fc7dd87656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:35.484364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30ca21-adb7-4f49-8d7b-ed519e3779e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:35.485878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30ca21-adb7-4f49-8d7b-ed519e3779e5 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-db30ca21-adb7-4f49-8d7b-ed519e3779e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:35.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:35.522985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:35.523858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:35.523858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:35.523858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:35.523858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:35.523858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:35.523858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:35.523858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:35.523858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:35.523858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 333/696] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:35.529551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30ca21-adb7-4f49-8d7b-ed519e3779e5 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:35.560348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead5b22-beba-4ba6-a81f-a0fc7dd87656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:35.562355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead5b22-beba-4ba6-a81f-a0fc7dd87656 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:35.590156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:35.732554 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30ca21-adb7-4f49-8d7b-ed519e3779e5 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:35.752806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 364/697] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:35.758629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead5b22-beba-4ba6-a81f-a0fc7dd87656 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:35.805108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd994dfb-b968-48e1-9445-56e5b10d60bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:35.806821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd994dfb-b968-48e1-9445-56e5b10d60bf None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:35.807406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd994dfb-b968-48e1-9445-56e5b10d60bf None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:35.807892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd994dfb-b968-48e1-9445-56e5b10d60bf None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:35.811759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd994dfb-b968-48e1-9445-56e5b10d60bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:35.812791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd994dfb-b968-48e1-9445-56e5b10d60bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:35.814921 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd994dfb-b968-48e1-9445-56e5b10d60bf None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:35.831152 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 334/698] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:35.832819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f6664-ac75-4d2d-8df0-e0e8b1dfbfb5 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:35.833427 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f6664-ac75-4d2d-8df0-e0e8b1dfbfb5 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:35.833993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f6664-ac75-4d2d-8df0-e0e8b1dfbfb5 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:35.835756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 335/699] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:35.843671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af594e-8bf0-4471-9d4f-d11d1c884f7f None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:35.843671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af594e-8bf0-4471-9d4f-d11d1c884f7f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:35.843671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af594e-8bf0-4471-9d4f-d11d1c884f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:35.865495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-65af594e-8bf0-4471-9d4f-d11d1c884f7f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:35.866910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-65af594e-8bf0-4471-9d4f-d11d1c884f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:35.870903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:35.902466 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead5b22-beba-4ba6-a81f-a0fc7dd87656 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:35.943041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:35.970101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 336/700] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:35:35.970101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af594e-8bf0-4471-9d4f-d11d1c884f7f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:35.970510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955daf3-5c98-4516-9cf3-01fd0f3d6317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:35.970510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955daf3-5c98-4516-9cf3-01fd0f3d6317 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:35.970510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955daf3-5c98-4516-9cf3-01fd0f3d6317 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:35.970510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955daf3-5c98-4516-9cf3-01fd0f3d6317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:35.995144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955daf3-5c98-4516-9cf3-01fd0f3d6317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:36.001576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955daf3-5c98-4516-9cf3-01fd0f3d6317 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-e955daf3-5c98-4516-9cf3-01fd0f3d6317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:36.025178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:36.026177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:36.027084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:36.027084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:36.027084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:36.027084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:36.027751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 365/701] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:36.028899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955daf3-5c98-4516-9cf3-01fd0f3d6317 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:36.125272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af594e-8bf0-4471-9d4f-d11d1c884f7f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:36.163806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955daf3-5c98-4516-9cf3-01fd0f3d6317 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:36.165266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73fd29-0bc3-4a9b-b4cf-ed866d624ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:36.166184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73fd29-0bc3-4a9b-b4cf-ed866d624ffd None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:36.166446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73fd29-0bc3-4a9b-b4cf-ed866d624ffd None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:36.166824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73fd29-0bc3-4a9b-b4cf-ed866d624ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:36.168797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73fd29-0bc3-4a9b-b4cf-ed866d624ffd None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=334a380fa68240c79a72746ee4a5f9dc) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:36.168797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73fd29-0bc3-4a9b-b4cf-ed866d624ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:36.170520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73fd29-0bc3-4a9b-b4cf-ed866d624ffd None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:36.177706 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 337/702] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:35 2022] GET /identity/v3/role_assignments?user.id=334a380fa68240c79a72746ee4a5f9dc&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:36.178646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73fd29-0bc3-4a9b-b4cf-ed866d624ffd None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:36.191827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:36.214058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b8607-9c11-4924-9697-7a4d054c0520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:36.216077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b8607-9c11-4924-9697-7a4d054c0520 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:36.216525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b8607-9c11-4924-9697-7a4d054c0520 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:36.216954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b8607-9c11-4924-9697-7a4d054c0520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:36.232244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b8607-9c11-4924-9697-7a4d054c0520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:36.234808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b8607-9c11-4924-9697-7a4d054c0520 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-1e3b8607-9c11-4924-9697-7a4d054c0520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:36.257635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:36.258759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:36.259811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:36.259811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:36.259811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:36.259811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:36.263245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 366/703] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:36.269946 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b8607-9c11-4924-9697-7a4d054c0520 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:36.305786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:36.407310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b8607-9c11-4924-9697-7a4d054c0520 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:36.433901 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73fd29-0bc3-4a9b-b4cf-ed866d624ffd None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:36.446534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbe18b-8cd9-472c-b529-facd21930c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:36.447208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbe18b-8cd9-472c-b529-facd21930c13 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:36.447492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbe18b-8cd9-472c-b529-facd21930c13 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:36.447767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbe18b-8cd9-472c-b529-facd21930c13 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:36.448504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbe18b-8cd9-472c-b529-facd21930c13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:36.449412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbe18b-8cd9-472c-b529-facd21930c13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:36.450774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbe18b-8cd9-472c-b529-facd21930c13 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:36.465378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 367/704] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:36.466752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4641a-41f4-41f7-955e-d502a41361d2 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:36.467049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4641a-41f4-41f7-955e-d502a41361d2 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:36.467330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4641a-41f4-41f7-955e-d502a41361d2 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:36.468876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 368/705] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:36.470589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7136b2-1c0c-4c52-ad7b-d7772d08d176 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:36.470937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7136b2-1c0c-4c52-ad7b-d7772d08d176 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:36.471348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7136b2-1c0c-4c52-ad7b-d7772d08d176 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:36.477906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 369/706] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:36.478650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7136b2-1c0c-4c52-ad7b-d7772d08d176 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:36.482259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09afaf15-52b2-4929-b3ff-89b70c2cbc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:36.483914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afaf15-52b2-4929-b3ff-89b70c2cbc6f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:36.484211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afaf15-52b2-4929-b3ff-89b70c2cbc6f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:36.484482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afaf15-52b2-4929-b3ff-89b70c2cbc6f None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:36.485232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09afaf15-52b2-4929-b3ff-89b70c2cbc6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:36.485571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09afaf15-52b2-4929-b3ff-89b70c2cbc6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:36.487083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09afaf15-52b2-4929-b3ff-89b70c2cbc6f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:36.502505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 338/707] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:36.503814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13450f-53eb-4c15-b244-2cd91d234bc0 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:36.503930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13450f-53eb-4c15-b244-2cd91d234bc0 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:36.504163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13450f-53eb-4c15-b244-2cd91d234bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:36.522319 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-eb13450f-53eb-4c15-b244-2cd91d234bc0 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:36.523441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-eb13450f-53eb-4c15-b244-2cd91d234bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:36.586645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:36.587896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:36.598149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 339/708] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:36.599854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7382c-9d7b-40ba-afd7-ac950f166217 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:36.600143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7382c-9d7b-40ba-afd7-ac950f166217 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:36.600418 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7382c-9d7b-40ba-afd7-ac950f166217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:36.610899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7136b2-1c0c-4c52-ad7b-d7772d08d176 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:36.621917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-b5d7382c-9d7b-40ba-afd7-ac950f166217 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:36.623118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b5d7382c-9d7b-40ba-afd7-ac950f166217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:36.654406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789e34e-5409-4b43-ad7b-7a98e7471946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:36.655230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789e34e-5409-4b43-ad7b-7a98e7471946 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:36.655629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789e34e-5409-4b43-ad7b-7a98e7471946 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:36.655908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789e34e-5409-4b43-ad7b-7a98e7471946 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:36.656706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789e34e-5409-4b43-ad7b-7a98e7471946 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:36.658393 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789e34e-5409-4b43-ad7b-7a98e7471946 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:36.689399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 370/709] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:35:36 2022] POST /identity/v3/users => generated 254 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:36.689399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789e34e-5409-4b43-ad7b-7a98e7471946 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:36.713567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 340/710] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:36.714651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7382c-9d7b-40ba-afd7-ac950f166217 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:36.833801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789e34e-5409-4b43-ad7b-7a98e7471946 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:36.852234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7382c-9d7b-40ba-afd7-ac950f166217 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:36.883696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc13e6d-0a95-4c9a-baf4-7674cbbabe7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:36.884610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc13e6d-0a95-4c9a-baf4-7674cbbabe7e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:36.884892 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc13e6d-0a95-4c9a-baf4-7674cbbabe7e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:36.885163 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc13e6d-0a95-4c9a-baf4-7674cbbabe7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:36.888342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:36.896413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc13e6d-0a95-4c9a-baf4-7674cbbabe7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:36.898171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc13e6d-0a95-4c9a-baf4-7674cbbabe7e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:36.900607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b2c11-4364-4821-9514-02bdd7a58b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:36.902368 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b2c11-4364-4821-9514-02bdd7a58b2a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:36.902484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b2c11-4364-4821-9514-02bdd7a58b2a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:36.902702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b2c11-4364-4821-9514-02bdd7a58b2a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:36.903368 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b2c11-4364-4821-9514-02bdd7a58b2a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:36.904754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b2c11-4364-4821-9514-02bdd7a58b2a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-7fc13e6d-0a95-4c9a-baf4-7674cbbabe7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:36.914551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:36.915475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:36.916385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:36.916385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:36.916385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:36.916385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:36.916385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:36.916385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:36.916385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:36.916385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:36.919353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 371/711] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:36.920235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc13e6d-0a95-4c9a-baf4-7674cbbabe7e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-340b2c11-4364-4821-9514-02bdd7a58b2a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:36.920525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:36.921610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:36.922656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 341/712] 10.222.0.70 () {56 vars in 1129 bytes} [Tue Jul 19 20:35:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:36.925724 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b2c11-4364-4821-9514-02bdd7a58b2a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:36.928402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:37.058291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc13e6d-0a95-4c9a-baf4-7674cbbabe7e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:37.080694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b2c11-4364-4821-9514-02bdd7a58b2a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:37.096320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a4b4e-56ae-4024-8ac3-43e9a63ebee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:37.097343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a4b4e-56ae-4024-8ac3-43e9a63ebee1 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:37.097859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a4b4e-56ae-4024-8ac3-43e9a63ebee1 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:37.098317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a4b4e-56ae-4024-8ac3-43e9a63ebee1 None admin] PATH_INFO: `/v3/services/network` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:37.099324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a4b4e-56ae-4024-8ac3-43e9a63ebee1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:37.101351 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a4b4e-56ae-4024-8ac3-43e9a63ebee1 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-a73a4b4e-56ae-4024-8ac3-43e9a63ebee1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:37.117583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:37.118945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:37.120442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 372/713] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:36 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:37.122805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411ac9e-2f34-4ee6-9865-8a10e95efce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:37.123039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a4b4e-56ae-4024-8ac3-43e9a63ebee1 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:37.123255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411ac9e-2f34-4ee6-9865-8a10e95efce7 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:37.123255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411ac9e-2f34-4ee6-9865-8a10e95efce7 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:37.123255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411ac9e-2f34-4ee6-9865-8a10e95efce7 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:37.123255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411ac9e-2f34-4ee6-9865-8a10e95efce7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:37.123255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411ac9e-2f34-4ee6-9865-8a10e95efce7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:37.124225 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411ac9e-2f34-4ee6-9865-8a10e95efce7 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:37.142597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 342/714] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:37.143632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411ac9e-2f34-4ee6-9865-8a10e95efce7 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:37.201233 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:37.205782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:37.247004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a4b4e-56ae-4024-8ac3-43e9a63ebee1 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:37.264234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411ac9e-2f34-4ee6-9865-8a10e95efce7 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:37.284246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103f4e6-b7db-4823-9563-0efbc98e85ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:37.284921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103f4e6-b7db-4823-9563-0efbc98e85ae None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:37.285203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103f4e6-b7db-4823-9563-0efbc98e85ae None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:37.285507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103f4e6-b7db-4823-9563-0efbc98e85ae None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:37.286262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103f4e6-b7db-4823-9563-0efbc98e85ae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:37.287845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103f4e6-b7db-4823-9563-0efbc98e85ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:37.289529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103f4e6-b7db-4823-9563-0efbc98e85ae None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:37.295216 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 373/715] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jul 19 20:35:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:37.296083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103f4e6-b7db-4823-9563-0efbc98e85ae None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:37.306684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98030680-5fd2-428d-83bd-46656d14d5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:37.307750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98030680-5fd2-428d-83bd-46656d14d5a1 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:37.308840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98030680-5fd2-428d-83bd-46656d14d5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:37.309329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98030680-5fd2-428d-83bd-46656d14d5a1 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:37.310377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98030680-5fd2-428d-83bd-46656d14d5a1 None admin] RBAC: Adding query filter params (name=network) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:37.310869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98030680-5fd2-428d-83bd-46656d14d5a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:37.312651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98030680-5fd2-428d-83bd-46656d14d5a1 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:37.319190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 343/716] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:35:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:37.321175 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98030680-5fd2-428d-83bd-46656d14d5a1 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:37.425707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103f4e6-b7db-4823-9563-0efbc98e85ae None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:37.458997 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:37.465879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b803641-e20e-4801-aacb-6f03dbd74d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:37.467015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b803641-e20e-4801-aacb-6f03dbd74d5c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:37.467711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b803641-e20e-4801-aacb-6f03dbd74d5c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:37.468052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b803641-e20e-4801-aacb-6f03dbd74d5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:37.469031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b803641-e20e-4801-aacb-6f03dbd74d5c None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=e1d09e168fc54f1da06b719989ae8ef0) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:37.469512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b803641-e20e-4801-aacb-6f03dbd74d5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:37.470417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98030680-5fd2-428d-83bd-46656d14d5a1 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:37.471163 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b803641-e20e-4801-aacb-6f03dbd74d5c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:37.485400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 374/717] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:37 2022] GET /identity/v3/role_assignments?user.id=e1d09e168fc54f1da06b719989ae8ef0&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=85420c68300642aabfd730fb8fef8347 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:37.485400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b803641-e20e-4801-aacb-6f03dbd74d5c None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:37.518440 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:37.574676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770d1d2-83bf-43f4-8931-911c747ed96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:37.575368 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770d1d2-83bf-43f4-8931-911c747ed96c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:37.575644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770d1d2-83bf-43f4-8931-911c747ed96c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:37.575913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770d1d2-83bf-43f4-8931-911c747ed96c None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:37.576634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770d1d2-83bf-43f4-8931-911c747ed96c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:37.576939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770d1d2-83bf-43f4-8931-911c747ed96c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:37.578371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770d1d2-83bf-43f4-8931-911c747ed96c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:37.595483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 344/718] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jul 19 20:35:37 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:37.710704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b803641-e20e-4801-aacb-6f03dbd74d5c None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:37.763645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c1203-b364-4baf-b11c-6cb96e8ecb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:37.764341 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1203-b364-4baf-b11c-6cb96e8ecb72 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:37.764628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1203-b364-4baf-b11c-6cb96e8ecb72 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:37.764908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1203-b364-4baf-b11c-6cb96e8ecb72 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:37.765665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c1203-b364-4baf-b11c-6cb96e8ecb72 None admin] RBAC: Adding query filter params (type=network) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:37.765975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c1203-b364-4baf-b11c-6cb96e8ecb72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:37.767546 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c1203-b364-4baf-b11c-6cb96e8ecb72 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:37.778935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 375/719] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:35:37 2022] GET /identity/v3/services?type=network => generated 124 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:38.694047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf1c1b-9dff-4900-9156-eea063d7173c None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:38.694555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf1c1b-9dff-4900-9156-eea063d7173c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:38.694555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf1c1b-9dff-4900-9156-eea063d7173c None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:38.696157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 345/720] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:38.707158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f4d06-30e8-4181-9e23-6e3bffd4ec57 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:38.707652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f4d06-30e8-4181-9e23-6e3bffd4ec57 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:38.707751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f4d06-30e8-4181-9e23-6e3bffd4ec57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:38.731032 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-661f4d06-30e8-4181-9e23-6e3bffd4ec57 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:38.732264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-661f4d06-30e8-4181-9e23-6e3bffd4ec57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:38.809057 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:38.831472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 376/721] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:38.878772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf1c1b-9dff-4900-9156-eea063d7173c None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:39.015101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf1c1b-9dff-4900-9156-eea063d7173c None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:39.059538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf46ea0-74fb-4763-b526-686baf4a41bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:39.060346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf46ea0-74fb-4763-b526-686baf4a41bd None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:39.060636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf46ea0-74fb-4763-b526-686baf4a41bd None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:39.060917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf46ea0-74fb-4763-b526-686baf4a41bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:39.077657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf46ea0-74fb-4763-b526-686baf4a41bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-4bf46ea0-74fb-4763-b526-686baf4a41bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:39.092292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:39.093283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:39.094622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 346/722] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:39.099869 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f4d06-30e8-4181-9e23-6e3bffd4ec57 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:39.212688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f4d06-30e8-4181-9e23-6e3bffd4ec57 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:39.249445 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fb215-63b3-4acd-a75b-9cc2a0b5a097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:39.250134 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fb215-63b3-4acd-a75b-9cc2a0b5a097 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:39.250423 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fb215-63b3-4acd-a75b-9cc2a0b5a097 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:39.250702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fb215-63b3-4acd-a75b-9cc2a0b5a097 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:39.251475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fb215-63b3-4acd-a75b-9cc2a0b5a097 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:39.251786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fb215-63b3-4acd-a75b-9cc2a0b5a097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:39.253165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fb215-63b3-4acd-a75b-9cc2a0b5a097 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:39.266199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 377/723] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:39.271663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf46ea0-74fb-4763-b526-686baf4a41bd None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:39.289192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f3638-8320-425e-819b-533e4ae80e80 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:39.289559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f3638-8320-425e-819b-533e4ae80e80 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:39.289791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f3638-8320-425e-819b-533e4ae80e80 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:39.291340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 378/724] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:39.297336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc048-6a69-452c-9c8c-eb4b52eb7884 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:39.298025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc048-6a69-452c-9c8c-eb4b52eb7884 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:39.298593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc048-6a69-452c-9c8c-eb4b52eb7884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:39.317522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0f1bc048-6a69-452c-9c8c-eb4b52eb7884 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:39.319373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0f1bc048-6a69-452c-9c8c-eb4b52eb7884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:39.373116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:39.379167 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:39.410253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 379/725] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:35:39.433713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bc048-6a69-452c-9c8c-eb4b52eb7884 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:39.558699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf46ea0-74fb-4763-b526-686baf4a41bd None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:39.570550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bc048-6a69-452c-9c8c-eb4b52eb7884 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:39.611698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c49ee2-dc80-4ce2-85a1-078eb9d0cc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:39.613102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c49ee2-dc80-4ce2-85a1-078eb9d0cc6d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:39.613561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c49ee2-dc80-4ce2-85a1-078eb9d0cc6d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:39.613989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c49ee2-dc80-4ce2-85a1-078eb9d0cc6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:39.626470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c49ee2-dc80-4ce2-85a1-078eb9d0cc6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-c4c49ee2-dc80-4ce2-85a1-078eb9d0cc6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:39.635175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:39.636342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:39.637382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:39.644072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 380/726] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:39.646335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c49ee2-dc80-4ce2-85a1-078eb9d0cc6d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:39.660546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9635ef8-fb42-41a5-b195-0b827de28543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:39.661514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9635ef8-fb42-41a5-b195-0b827de28543 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:39.661806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9635ef8-fb42-41a5-b195-0b827de28543 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:39.662088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9635ef8-fb42-41a5-b195-0b827de28543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:39.686508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9635ef8-fb42-41a5-b195-0b827de28543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:39.687991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9635ef8-fb42-41a5-b195-0b827de28543 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:39.696000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-f9635ef8-fb42-41a5-b195-0b827de28543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:39.728826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:39.730872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:39.731913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:39.731913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:39.731913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:39.731913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:39.731913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 347/727] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:39.740472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9635ef8-fb42-41a5-b195-0b827de28543 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:39.765802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c49ee2-dc80-4ce2-85a1-078eb9d0cc6d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:39.816248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f0dfe-93a4-4ad5-82e6-9ce746efefad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:39.817211 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0dfe-93a4-4ad5-82e6-9ce746efefad None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:39.817693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0dfe-93a4-4ad5-82e6-9ce746efefad None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:39.818194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0dfe-93a4-4ad5-82e6-9ce746efefad None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:39.819125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f0dfe-93a4-4ad5-82e6-9ce746efefad None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:39.819683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f0dfe-93a4-4ad5-82e6-9ce746efefad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:39.821407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f0dfe-93a4-4ad5-82e6-9ce746efefad None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:39.840620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 381/728] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:39.846496 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f0dfe-93a4-4ad5-82e6-9ce746efefad None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:39.874495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:39.887361 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9635ef8-fb42-41a5-b195-0b827de28543 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:39.927458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f555c-0b95-4c83-a0ea-7f70f0a9f039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:39.929306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f555c-0b95-4c83-a0ea-7f70f0a9f039 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:39.929608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f555c-0b95-4c83-a0ea-7f70f0a9f039 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:39.929932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f555c-0b95-4c83-a0ea-7f70f0a9f039 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:39.931079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f555c-0b95-4c83-a0ea-7f70f0a9f039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:39.931389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f555c-0b95-4c83-a0ea-7f70f0a9f039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:39.932766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f555c-0b95-4c83-a0ea-7f70f0a9f039 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:39.945487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 348/729] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:39.951545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea246455-bada-41ee-ba65-c4c71b8635d5 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:39.951545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea246455-bada-41ee-ba65-c4c71b8635d5 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:39.951545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea246455-bada-41ee-ba65-c4c71b8635d5 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:39.954038 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 349/730] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:39.955315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea246455-bada-41ee-ba65-c4c71b8635d5 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:40.010007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f0dfe-93a4-4ad5-82e6-9ce746efefad None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:40.014993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:40.050976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ef734-1eca-417c-96d4-6c4f00be036a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:40.052205 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ef734-1eca-417c-96d4-6c4f00be036a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.052487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ef734-1eca-417c-96d4-6c4f00be036a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.052758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ef734-1eca-417c-96d4-6c4f00be036a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.070848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ef734-1eca-417c-96d4-6c4f00be036a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:40.072556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ef734-1eca-417c-96d4-6c4f00be036a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-5a6ef734-1eca-417c-96d4-6c4f00be036a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:40.097297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:40.098275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:40.099145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:40.099145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:40.099145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:40.099145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:40.101882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 382/731] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:40.103903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0e3aa-4bdd-488e-850f-3f5754445394 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.104259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea246455-bada-41ee-ba65-c4c71b8635d5 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:40.104540 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0e3aa-4bdd-488e-850f-3f5754445394 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.105006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0e3aa-4bdd-488e-850f-3f5754445394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.124638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-fde0e3aa-4bdd-488e-850f-3f5754445394 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:40.126264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fde0e3aa-4bdd-488e-850f-3f5754445394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:40.149272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb4e11-8dd3-49a1-813c-efd056526174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:40.150947 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb4e11-8dd3-49a1-813c-efd056526174 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.151394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb4e11-8dd3-49a1-813c-efd056526174 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.152657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb4e11-8dd3-49a1-813c-efd056526174 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.163651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb4e11-8dd3-49a1-813c-efd056526174 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:40.166835 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb4e11-8dd3-49a1-813c-efd056526174 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-35eb4e11-8dd3-49a1-813c-efd056526174 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:40.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:40.189540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:40.190700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:40.190700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:40.190700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:40.190700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:40.190700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:40.190700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:40.190700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:40.190700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:40.190700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:40.190700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:40.197780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 350/732] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:40.197780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb4e11-8dd3-49a1-813c-efd056526174 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:40.216601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:40.242884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 383/733] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:40.242884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0e3aa-4bdd-488e-850f-3f5754445394 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:40.340413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb4e11-8dd3-49a1-813c-efd056526174 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:40.371605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:40.385161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615a6d4-41cd-44f2-ae3e-d9c42817b35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:40.385735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615a6d4-41cd-44f2-ae3e-d9c42817b35b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.386840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615a6d4-41cd-44f2-ae3e-d9c42817b35b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.386840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615a6d4-41cd-44f2-ae3e-d9c42817b35b None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.387549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615a6d4-41cd-44f2-ae3e-d9c42817b35b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:40.387878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615a6d4-41cd-44f2-ae3e-d9c42817b35b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:40.389271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615a6d4-41cd-44f2-ae3e-d9c42817b35b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:40.399501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0e3aa-4bdd-488e-850f-3f5754445394 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:40.410097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 351/734] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:40.410097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615a6d4-41cd-44f2-ae3e-d9c42817b35b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:40.445899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dc190-3e7e-42ec-9a94-11b08c09393e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:40.446840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dc190-3e7e-42ec-9a94-11b08c09393e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.447052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dc190-3e7e-42ec-9a94-11b08c09393e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.447309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dc190-3e7e-42ec-9a94-11b08c09393e None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.448237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dc190-3e7e-42ec-9a94-11b08c09393e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:40.449861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dc190-3e7e-42ec-9a94-11b08c09393e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:40.453125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dc190-3e7e-42ec-9a94-11b08c09393e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:40.475264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 384/735] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jul 19 20:35:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:40.477154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e069408-b200-4f82-8ad2-f4f5d667fb16 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.477468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e069408-b200-4f82-8ad2-f4f5d667fb16 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.477751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e069408-b200-4f82-8ad2-f4f5d667fb16 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.483266 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 385/736] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:40.483266 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e069408-b200-4f82-8ad2-f4f5d667fb16 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:40.497738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:40.537089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615a6d4-41cd-44f2-ae3e-d9c42817b35b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:40.580759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bd5bf-d860-44d1-8769-6359fdc1b35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:40.581622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bd5bf-d860-44d1-8769-6359fdc1b35d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.582011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bd5bf-d860-44d1-8769-6359fdc1b35d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.582226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bd5bf-d860-44d1-8769-6359fdc1b35d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.596078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bd5bf-d860-44d1-8769-6359fdc1b35d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-aa3bd5bf-d860-44d1-8769-6359fdc1b35d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:40.604125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:40.605184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:40.606092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 352/737] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:40.606767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bd5bf-d860-44d1-8769-6359fdc1b35d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:40.612218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e069408-b200-4f82-8ad2-f4f5d667fb16 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:40.650114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426eefc-fab0-4658-9ef7-52b14c0ccc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:40.650929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426eefc-fab0-4658-9ef7-52b14c0ccc4b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.651226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426eefc-fab0-4658-9ef7-52b14c0ccc4b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.651508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426eefc-fab0-4658-9ef7-52b14c0ccc4b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.660280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426eefc-fab0-4658-9ef7-52b14c0ccc4b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:40.661854 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426eefc-fab0-4658-9ef7-52b14c0ccc4b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-2426eefc-fab0-4658-9ef7-52b14c0ccc4b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:40.678348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:40.679507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:40.680744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:40.680744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:40.680744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:40.680744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:40.680744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:40.680744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:40.680744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:40.680744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:40.680744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:35:40.680744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:40.680744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 386/738] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jul 19 20:35:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:40.681326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b6fad-25ee-449c-9678-668138669474 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.681326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b6fad-25ee-449c-9678-668138669474 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.681603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b6fad-25ee-449c-9678-668138669474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.687759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:40.699766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-4d0b6fad-25ee-449c-9678-668138669474 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:40.700943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4d0b6fad-25ee-449c-9678-668138669474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:40.720712 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bd5bf-d860-44d1-8769-6359fdc1b35d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:40.746770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:40.755974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2118e-984c-483c-bfa5-c53ac30003ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:40.756690 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2118e-984c-483c-bfa5-c53ac30003ef None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.756910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2118e-984c-483c-bfa5-c53ac30003ef None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.757192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2118e-984c-483c-bfa5-c53ac30003ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.768164 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 387/739] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:40.769004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b6fad-25ee-449c-9678-668138669474 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:40.839186 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2118e-984c-483c-bfa5-c53ac30003ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:40.841939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2118e-984c-483c-bfa5-c53ac30003ef None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:40.887651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b6fad-25ee-449c-9678-668138669474 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:40.935803 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d2060-8baf-458b-bd2b-f8cce31a82db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:40.938203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d2060-8baf-458b-bd2b-f8cce31a82db None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.938203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d2060-8baf-458b-bd2b-f8cce31a82db None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.938203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d2060-8baf-458b-bd2b-f8cce31a82db None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.938203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d2060-8baf-458b-bd2b-f8cce31a82db None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:40.938203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d2060-8baf-458b-bd2b-f8cce31a82db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:40.939648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d2060-8baf-458b-bd2b-f8cce31a82db None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:40.941639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:40.950527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 388/740] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:40.951712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80ea1a-7fb0-420e-9555-c8c44f7b9f1b None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:40.952027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80ea1a-7fb0-420e-9555-c8c44f7b9f1b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:40.952315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80ea1a-7fb0-420e-9555-c8c44f7b9f1b None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:40.953601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 389/741] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:40.954332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80ea1a-7fb0-420e-9555-c8c44f7b9f1b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:40.964965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 353/742] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:40.966163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2118e-984c-483c-bfa5-c53ac30003ef None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:40.993109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:41.065866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80ea1a-7fb0-420e-9555-c8c44f7b9f1b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:41.068593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2118e-984c-483c-bfa5-c53ac30003ef None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:41.099430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47531ea-db05-4dc7-b777-67d8247ae537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:41.100099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47531ea-db05-4dc7-b777-67d8247ae537 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:41.100380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47531ea-db05-4dc7-b777-67d8247ae537 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:41.100665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47531ea-db05-4dc7-b777-67d8247ae537 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:41.101537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47531ea-db05-4dc7-b777-67d8247ae537 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:41.101845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47531ea-db05-4dc7-b777-67d8247ae537 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:41.102220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d17538-3105-4dc9-8c40-ec8f4b5e6010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:41.102893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d17538-3105-4dc9-8c40-ec8f4b5e6010 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:41.103182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d17538-3105-4dc9-8c40-ec8f4b5e6010 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:41.103277 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47531ea-db05-4dc7-b777-67d8247ae537 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:41.103463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d17538-3105-4dc9-8c40-ec8f4b5e6010 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:41.104203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d17538-3105-4dc9-8c40-ec8f4b5e6010 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:41.106105 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d17538-3105-4dc9-8c40-ec8f4b5e6010 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:41.123219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 354/743] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:35:40 2022] POST /identity/v3/services => generated 251 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:41.123413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 390/744] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jul 19 20:35:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:41.124319 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872156cb-787e-4c02-871d-6d392cc0cb6a None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:41.124614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872156cb-787e-4c02-871d-6d392cc0cb6a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:41.124896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872156cb-787e-4c02-871d-6d392cc0cb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:41.126509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d17538-3105-4dc9-8c40-ec8f4b5e6010 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:41.155206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-872156cb-787e-4c02-871d-6d392cc0cb6a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:41.156384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-872156cb-787e-4c02-871d-6d392cc0cb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:41.217038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:41.238194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:41.244844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 391/745] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:41 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:35:41.246350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872156cb-787e-4c02-871d-6d392cc0cb6a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:41.266075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d17538-3105-4dc9-8c40-ec8f4b5e6010 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:41.314064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ded9a2-831f-46b4-97c8-5f7a2d177bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:41.314883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ded9a2-831f-46b4-97c8-5f7a2d177bee None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:41.315160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ded9a2-831f-46b4-97c8-5f7a2d177bee None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:41.315433 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ded9a2-831f-46b4-97c8-5f7a2d177bee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:41.327516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ded9a2-831f-46b4-97c8-5f7a2d177bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:41.329268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ded9a2-831f-46b4-97c8-5f7a2d177bee None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-66ded9a2-831f-46b4-97c8-5f7a2d177bee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:41.348124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:41.349183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:41.350321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:41.350321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:41.350321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:41.350321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:41.352234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 355/746] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:41.353260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ded9a2-831f-46b4-97c8-5f7a2d177bee None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:41.378254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872156cb-787e-4c02-871d-6d392cc0cb6a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:41.415368 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b87184-3ff1-493a-b5bb-0558ca75c1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:41.416085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b87184-3ff1-493a-b5bb-0558ca75c1a7 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:41.416371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b87184-3ff1-493a-b5bb-0558ca75c1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:41.416642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b87184-3ff1-493a-b5bb-0558ca75c1a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:41.428699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b87184-3ff1-493a-b5bb-0558ca75c1a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:41.430432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b87184-3ff1-493a-b5bb-0558ca75c1a7 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-09b87184-3ff1-493a-b5bb-0558ca75c1a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:41.450847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:41.452092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:41.453342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:41.453342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:41.453342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:41.453342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:41.453342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 392/747] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:41.454932 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b87184-3ff1-493a-b5bb-0558ca75c1a7 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:41.484443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ded9a2-831f-46b4-97c8-5f7a2d177bee None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:41.499318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:41.525272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20877f6f-a191-46cb-83d4-c96ed9b9da5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:41.526060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20877f6f-a191-46cb-83d4-c96ed9b9da5c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:41.526349 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20877f6f-a191-46cb-83d4-c96ed9b9da5c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:41.527148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20877f6f-a191-46cb-83d4-c96ed9b9da5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:41.538872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:41.578214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b87184-3ff1-493a-b5bb-0558ca75c1a7 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:41.602494 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20877f6f-a191-46cb-83d4-c96ed9b9da5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:41.603956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20877f6f-a191-46cb-83d4-c96ed9b9da5c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:41.615562 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f8f53-b22f-4d8a-8ea5-e9f01f79d442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:41.616247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f8f53-b22f-4d8a-8ea5-e9f01f79d442 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:41.616534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f8f53-b22f-4d8a-8ea5-e9f01f79d442 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:41.616817 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f8f53-b22f-4d8a-8ea5-e9f01f79d442 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:41.617582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f8f53-b22f-4d8a-8ea5-e9f01f79d442 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:41.618979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f8f53-b22f-4d8a-8ea5-e9f01f79d442 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:41.645758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 393/748] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:35:41 2022] POST /identity/v3/services => generated 240 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:41.646597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f8f53-b22f-4d8a-8ea5-e9f01f79d442 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:41.735886 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 356/749] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:41 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:35:41.737244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20877f6f-a191-46cb-83d4-c96ed9b9da5c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:41.769973 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:41.788920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:41.796966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f8f53-b22f-4d8a-8ea5-e9f01f79d442 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:41.840930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddf4aa-1eda-4aa8-b904-d23312a1e900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:41.841734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddf4aa-1eda-4aa8-b904-d23312a1e900 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:41.841968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddf4aa-1eda-4aa8-b904-d23312a1e900 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:41.842253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddf4aa-1eda-4aa8-b904-d23312a1e900 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:41.843002 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddf4aa-1eda-4aa8-b904-d23312a1e900 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:41.843316 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddf4aa-1eda-4aa8-b904-d23312a1e900 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:41.844721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddf4aa-1eda-4aa8-b904-d23312a1e900 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:41.857824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 394/750] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:41.858549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddf4aa-1eda-4aa8-b904-d23312a1e900 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:41.867101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20877f6f-a191-46cb-83d4-c96ed9b9da5c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:41.926018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc2399-a12d-4d99-845c-05b825363c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:41.927103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc2399-a12d-4d99-845c-05b825363c8f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:41.927591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc2399-a12d-4d99-845c-05b825363c8f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:41.928076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc2399-a12d-4d99-845c-05b825363c8f None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:41.929011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc2399-a12d-4d99-845c-05b825363c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:41.929563 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc2399-a12d-4d99-845c-05b825363c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:41.931123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc2399-a12d-4d99-845c-05b825363c8f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:41.945744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 357/751] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:41.946664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc2399-a12d-4d99-845c-05b825363c8f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:41.987116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddf4aa-1eda-4aa8-b904-d23312a1e900 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:42.034898 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2a6bb-9e8a-47b4-bffe-708c40540ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:42.036155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2a6bb-9e8a-47b4-bffe-708c40540ceb None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:42.036618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2a6bb-9e8a-47b4-bffe-708c40540ceb None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:42.037069 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2a6bb-9e8a-47b4-bffe-708c40540ceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:42.049140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2a6bb-9e8a-47b4-bffe-708c40540ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:42.050940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2a6bb-9e8a-47b4-bffe-708c40540ceb None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:42.062469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-32f2a6bb-9e8a-47b4-bffe-708c40540ceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:42.074836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:42.075769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:42.076639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:42.076639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:42.076639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:42.076639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:42.081750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:42.084080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 395/752] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:42.086140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2a6bb-9e8a-47b4-bffe-708c40540ceb None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:42.086622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc2399-a12d-4d99-845c-05b825363c8f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:42.127989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6de8d-7231-4c11-83c7-6c5d8c76c6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:42.128985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6de8d-7231-4c11-83c7-6c5d8c76c6a0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:42.129498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6de8d-7231-4c11-83c7-6c5d8c76c6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:42.129963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6de8d-7231-4c11-83c7-6c5d8c76c6a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:42.139771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6de8d-7231-4c11-83c7-6c5d8c76c6a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:42.141980 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a6de8d-7231-4c11-83c7-6c5d8c76c6a0 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-23a6de8d-7231-4c11-83c7-6c5d8c76c6a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:42.162819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:42.163995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:42.165138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:42.165138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:42.165138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:42.165138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:42.165138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:42.165138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:42.165138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:42.165138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:42.165138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:42.165138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:42.169711 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 358/753] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jul 19 20:35:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:42.172145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6de8d-7231-4c11-83c7-6c5d8c76c6a0 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:42.230620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2a6bb-9e8a-47b4-bffe-708c40540ceb None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:42.274960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fe48b-5f7a-4322-bc1a-bde674c334d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:42.275872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fe48b-5f7a-4322-bc1a-bde674c334d4 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:42.276257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fe48b-5f7a-4322-bc1a-bde674c334d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:42.276681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fe48b-5f7a-4322-bc1a-bde674c334d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:42.285208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5fe48b-5f7a-4322-bc1a-bde674c334d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:42.288069 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5fe48b-5f7a-4322-bc1a-bde674c334d4 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:42.299133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6de8d-7231-4c11-83c7-6c5d8c76c6a0 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-bc5fe48b-5f7a-4322-bc1a-bde674c334d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:42.305110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:42.306220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:42.307281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:42.307281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:42.307281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:42.307281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:42.307281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:42.307281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:42.307281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:42.307281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:42.309597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 396/754] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:42.310553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fe48b-5f7a-4322-bc1a-bde674c334d4 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:42.345876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449570b-fccd-415f-b6a7-587dee21447e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:42.346669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449570b-fccd-415f-b6a7-587dee21447e None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:42.347075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449570b-fccd-415f-b6a7-587dee21447e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:42.347380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449570b-fccd-415f-b6a7-587dee21447e None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:42.348470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449570b-fccd-415f-b6a7-587dee21447e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:42.348470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449570b-fccd-415f-b6a7-587dee21447e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:42.351925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449570b-fccd-415f-b6a7-587dee21447e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:42.389436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:42.389630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 359/755] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:42.391660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449570b-fccd-415f-b6a7-587dee21447e None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:42.432671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:42.444293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fe48b-5f7a-4322-bc1a-bde674c334d4 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:42.502276 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d46cc-1910-4bac-91ef-8a5f2e899b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:42.503940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d46cc-1910-4bac-91ef-8a5f2e899b3d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:42.504053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d46cc-1910-4bac-91ef-8a5f2e899b3d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:42.504626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d46cc-1910-4bac-91ef-8a5f2e899b3d None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:42.506445 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d46cc-1910-4bac-91ef-8a5f2e899b3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:42.507096 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d46cc-1910-4bac-91ef-8a5f2e899b3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:42.509924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d46cc-1910-4bac-91ef-8a5f2e899b3d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:42.544411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449570b-fccd-415f-b6a7-587dee21447e None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:42.544992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 397/756] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jul 19 20:35:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:42.546670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d46cc-1910-4bac-91ef-8a5f2e899b3d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:42.603708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce6399-53cf-409a-a08b-33d029ae7345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:42.605563 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce6399-53cf-409a-a08b-33d029ae7345 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:42.606127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce6399-53cf-409a-a08b-33d029ae7345 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:42.606704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce6399-53cf-409a-a08b-33d029ae7345 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:42.608331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce6399-53cf-409a-a08b-33d029ae7345 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:42.609045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce6399-53cf-409a-a08b-33d029ae7345 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:42.611900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce6399-53cf-409a-a08b-33d029ae7345 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:42.635548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 360/757] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:42.637561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce6399-53cf-409a-a08b-33d029ae7345 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:42.673396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d46cc-1910-4bac-91ef-8a5f2e899b3d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:42.697295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:42.719837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bea86d-483e-48aa-bd9f-0daef7886900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:42.720969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bea86d-483e-48aa-bd9f-0daef7886900 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:42.721459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bea86d-483e-48aa-bd9f-0daef7886900 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:42.721910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bea86d-483e-48aa-bd9f-0daef7886900 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:42.731201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bea86d-483e-48aa-bd9f-0daef7886900 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:42.733451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bea86d-483e-48aa-bd9f-0daef7886900 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-21bea86d-483e-48aa-bd9f-0daef7886900 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:42.750840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:42.751772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:42.752658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:42.752658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:42.752658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:42.752658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:42.752658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:42.752658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:42.752658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:42.752658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:42.760100 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 398/758] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:42.760100 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bea86d-483e-48aa-bd9f-0daef7886900 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:42.802041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:42.827809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce6399-53cf-409a-a08b-33d029ae7345 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:42.872774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0841706-a3e9-4abc-9617-1f950d9a7bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:42.873339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0841706-a3e9-4abc-9617-1f950d9a7bc5 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:42.873737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0841706-a3e9-4abc-9617-1f950d9a7bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:42.874121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0841706-a3e9-4abc-9617-1f950d9a7bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:42.886733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0841706-a3e9-4abc-9617-1f950d9a7bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:42.888197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0841706-a3e9-4abc-9617-1f950d9a7bc5 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:42.902110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bea86d-483e-48aa-bd9f-0daef7886900 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-d0841706-a3e9-4abc-9617-1f950d9a7bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:42.907353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:42.908230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:42.909102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:42.909102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:42.909102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:42.909102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:42.909102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 361/759] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:42.909806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0841706-a3e9-4abc-9617-1f950d9a7bc5 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:42.942666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ec3ca-2d55-420a-81d7-272a17d15715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:42.943730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ec3ca-2d55-420a-81d7-272a17d15715 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:42.944212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ec3ca-2d55-420a-81d7-272a17d15715 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:42.944685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ec3ca-2d55-420a-81d7-272a17d15715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:42.945773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ec3ca-2d55-420a-81d7-272a17d15715 None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=334a380fa68240c79a72746ee4a5f9dc) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:42.946289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ec3ca-2d55-420a-81d7-272a17d15715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:42.948172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ec3ca-2d55-420a-81d7-272a17d15715 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:42.956345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 399/760] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:42 2022] GET /identity/v3/role_assignments?user.id=334a380fa68240c79a72746ee4a5f9dc&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=85420c68300642aabfd730fb8fef8347 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:42.957805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ec3ca-2d55-420a-81d7-272a17d15715 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:43.017075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:43.094682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ec3ca-2d55-420a-81d7-272a17d15715 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:43.123070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0841706-a3e9-4abc-9617-1f950d9a7bc5 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:43.147173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ae550-00f2-4e83-8162-314e584d52f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:43.147859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ae550-00f2-4e83-8162-314e584d52f6 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:43.148140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ae550-00f2-4e83-8162-314e584d52f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:43.148416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ae550-00f2-4e83-8162-314e584d52f6 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:43.149152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ae550-00f2-4e83-8162-314e584d52f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:43.149485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ae550-00f2-4e83-8162-314e584d52f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:43.150907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ae550-00f2-4e83-8162-314e584d52f6 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:43.165664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 400/761] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:43.174495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ae550-00f2-4e83-8162-314e584d52f6 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:43.211064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:43.234349 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110d6c4-0d79-446f-bbc9-14dafa89168a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:43.235845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110d6c4-0d79-446f-bbc9-14dafa89168a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:43.236730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110d6c4-0d79-446f-bbc9-14dafa89168a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:43.245812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110d6c4-0d79-446f-bbc9-14dafa89168a None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:43.248507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110d6c4-0d79-446f-bbc9-14dafa89168a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:43.249304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110d6c4-0d79-446f-bbc9-14dafa89168a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:43.252934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110d6c4-0d79-446f-bbc9-14dafa89168a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:43.296486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ae550-00f2-4e83-8162-314e584d52f6 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:43.299825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 362/762] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:43.300545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:43.322443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110d6c4-0d79-446f-bbc9-14dafa89168a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:43.346680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca582d-16f3-4552-b888-831e43809b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:43.347836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca582d-16f3-4552-b888-831e43809b50 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:43.348240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca582d-16f3-4552-b888-831e43809b50 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:43.348615 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca582d-16f3-4552-b888-831e43809b50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:43.357255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca582d-16f3-4552-b888-831e43809b50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:43.358880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca582d-16f3-4552-b888-831e43809b50 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-51ca582d-16f3-4552-b888-831e43809b50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:43.379594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:43.380572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:43.381953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:43.381953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:43.381953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:43.381953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:43.381953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:43.381953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:43.381953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:43.381953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:43.384297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 401/763] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:43.390230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca582d-16f3-4552-b888-831e43809b50 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:43.513180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca582d-16f3-4552-b888-831e43809b50 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:43.575408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110d6c4-0d79-446f-bbc9-14dafa89168a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:43.599694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5129e-250a-4512-9ec1-b382a70d2515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:43.600300 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5129e-250a-4512-9ec1-b382a70d2515 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:43.600588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5129e-250a-4512-9ec1-b382a70d2515 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:43.600867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5129e-250a-4512-9ec1-b382a70d2515 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:43.601700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5129e-250a-4512-9ec1-b382a70d2515 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:43.602021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5129e-250a-4512-9ec1-b382a70d2515 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:43.603485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5129e-250a-4512-9ec1-b382a70d2515 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:43.630172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 402/764] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:43.634221 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05b474-904d-435c-a0a0-09325dd82972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:43.635010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b474-904d-435c-a0a0-09325dd82972 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:43.636881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5129e-250a-4512-9ec1-b382a70d2515 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:43.637891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b474-904d-435c-a0a0-09325dd82972 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:43.638293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b474-904d-435c-a0a0-09325dd82972 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:43.639785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05b474-904d-435c-a0a0-09325dd82972 None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=2711df504d424fa0b05cd16d7fd5acfb) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:43.641549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05b474-904d-435c-a0a0-09325dd82972 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:43.645266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05b474-904d-435c-a0a0-09325dd82972 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:43.663393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 363/765] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:43 2022] GET /identity/v3/role_assignments?user.id=2711df504d424fa0b05cd16d7fd5acfb&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 259 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:43.679776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:43.718866 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3793-d4ed-4ed8-a31a-d8d3e516e0d8 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:43.719216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3793-d4ed-4ed8-a31a-d8d3e516e0d8 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:43.719614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3793-d4ed-4ed8-a31a-d8d3e516e0d8 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:43.722168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 364/766] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:35:43.731352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f88bb-5332-4b8f-9dd6-4ac3367f903c None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:43.731816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f88bb-5332-4b8f-9dd6-4ac3367f903c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:43.732754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f88bb-5332-4b8f-9dd6-4ac3367f903c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:43.773066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-2e9f88bb-5332-4b8f-9dd6-4ac3367f903c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:43.776426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2e9f88bb-5332-4b8f-9dd6-4ac3367f903c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:43.851125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5129e-250a-4512-9ec1-b382a70d2515 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:43.901247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:43.913580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4099ddf-7de7-4fb8-a8a6-2cd074d1931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:43.915115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4099ddf-7de7-4fb8-a8a6-2cd074d1931f None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:43.915412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4099ddf-7de7-4fb8-a8a6-2cd074d1931f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:43.915696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4099ddf-7de7-4fb8-a8a6-2cd074d1931f None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/e1d09e168fc54f1da06b719989ae8ef0/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:43.918573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4099ddf-7de7-4fb8-a8a6-2cd074d1931f None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=e1d09e168fc54f1da06b719989ae8ef0, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:43.925415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4099ddf-7de7-4fb8-a8a6-2cd074d1931f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:43.958107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 403/767] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:35:43 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/e1d09e168fc54f1da06b719989ae8ef0/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:35:43.996389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 365/768] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:35:44.048620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4099ddf-7de7-4fb8-a8a6-2cd074d1931f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:44.166162 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:44.190388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4099ddf-7de7-4fb8-a8a6-2cd074d1931f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:44.239617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d9e42-b5fd-48f1-872e-8b4cc676751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:44.240507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9e42-b5fd-48f1-872e-8b4cc676751f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:44.240783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9e42-b5fd-48f1-872e-8b4cc676751f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:44.241115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9e42-b5fd-48f1-872e-8b4cc676751f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:44.241946 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d9e42-b5fd-48f1-872e-8b4cc676751f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:44.243774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d9e42-b5fd-48f1-872e-8b4cc676751f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-6e2d9e42-b5fd-48f1-872e-8b4cc676751f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:44.266678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:44.267783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:35:44.268660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:44.271945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 404/769] 10.222.0.70 () {56 vars in 1129 bytes} [Tue Jul 19 20:35:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:44.280307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f88bb-5332-4b8f-9dd6-4ac3367f903c None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:44.312414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdce4e-9e19-4101-903e-3ee04ef75a21 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:44.313267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdce4e-9e19-4101-903e-3ee04ef75a21 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:44.314001 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdce4e-9e19-4101-903e-3ee04ef75a21 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:44.316818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 405/770] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:44.327454 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becccc85-da7d-4139-89c2-1e62a071624f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:44.328265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becccc85-da7d-4139-89c2-1e62a071624f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:44.328969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becccc85-da7d-4139-89c2-1e62a071624f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:44.376602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-becccc85-da7d-4139-89c2-1e62a071624f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:44.379251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-becccc85-da7d-4139-89c2-1e62a071624f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:44.483547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f88bb-5332-4b8f-9dd6-4ac3367f903c None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:44.576578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 406/771] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:35:44.592935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab61b4f-87d6-4da9-9c14-67714d08a1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:44.594276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab61b4f-87d6-4da9-9c14-67714d08a1c9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:44.594719 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab61b4f-87d6-4da9-9c14-67714d08a1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:44.595149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab61b4f-87d6-4da9-9c14-67714d08a1c9 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:44.596572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab61b4f-87d6-4da9-9c14-67714d08a1c9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:44.597266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab61b4f-87d6-4da9-9c14-67714d08a1c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:44.600066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab61b4f-87d6-4da9-9c14-67714d08a1c9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:44.600887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becccc85-da7d-4139-89c2-1e62a071624f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:44.617198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 366/772] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jul 19 20:35:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:44.626883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab61b4f-87d6-4da9-9c14-67714d08a1c9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:44.744516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:44.811942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:44.830277 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becccc85-da7d-4139-89c2-1e62a071624f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:44.935667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d0cb5-816f-4be9-8000-fc3df05e4883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:44.936500 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d0cb5-816f-4be9-8000-fc3df05e4883 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:44.936823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d0cb5-816f-4be9-8000-fc3df05e4883 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:44.937133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d0cb5-816f-4be9-8000-fc3df05e4883 None admin] PATH_INFO: `/v3/services/network` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:44.937990 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d0cb5-816f-4be9-8000-fc3df05e4883 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:44.939732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d0cb5-816f-4be9-8000-fc3df05e4883 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-870d0cb5-816f-4be9-8000-fc3df05e4883 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:44.962987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:44.964185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:35:44.965157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:44.969962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 407/773] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:44 2022] GET /identity/v3/services/network => generated 88 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:44.971821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d0cb5-816f-4be9-8000-fc3df05e4883 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:44.973338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab61b4f-87d6-4da9-9c14-67714d08a1c9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:45.075248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fe081-c927-4662-9cc1-ecd94c6b8444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:45.076082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fe081-c927-4662-9cc1-ecd94c6b8444 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:45.076370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fe081-c927-4662-9cc1-ecd94c6b8444 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:45.076661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fe081-c927-4662-9cc1-ecd94c6b8444 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:45.077402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fe081-c927-4662-9cc1-ecd94c6b8444 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:45.077779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fe081-c927-4662-9cc1-ecd94c6b8444 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:45.079303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fe081-c927-4662-9cc1-ecd94c6b8444 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:45.091157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 367/774] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jul 19 20:35:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:45.097878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fe081-c927-4662-9cc1-ecd94c6b8444 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:45.135606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d0cb5-816f-4be9-8000-fc3df05e4883 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:45.179757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b2cc7-31d4-4533-bcad-449447920031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:45.180434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b2cc7-31d4-4533-bcad-449447920031 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:45.180715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b2cc7-31d4-4533-bcad-449447920031 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:45.180994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b2cc7-31d4-4533-bcad-449447920031 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:45.181734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b2cc7-31d4-4533-bcad-449447920031 None admin] RBAC: Adding query filter params (name=network) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:45.182040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b2cc7-31d4-4533-bcad-449447920031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:45.184696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b2cc7-31d4-4533-bcad-449447920031 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:45.196320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 408/775] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:35:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:45.204251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b2cc7-31d4-4533-bcad-449447920031 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:45.208756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:45.343270 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fe081-c927-4662-9cc1-ecd94c6b8444 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:45.360521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:45.395143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88603c7-6501-42e0-a213-37f1082334b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:45.395821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88603c7-6501-42e0-a213-37f1082334b9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:45.396107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88603c7-6501-42e0-a213-37f1082334b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:45.396461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88603c7-6501-42e0-a213-37f1082334b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:45.397586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88603c7-6501-42e0-a213-37f1082334b9 None admin] RBAC: Adding query filter params (interface=public, service_id=454c2183fa6041e291d324070e47d2f6, region_id=RegionOne) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:45.397927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88603c7-6501-42e0-a213-37f1082334b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:45.399695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88603c7-6501-42e0-a213-37f1082334b9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:45.419117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b2cc7-31d4-4533-bcad-449447920031 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:45.424839 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 368/776] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jul 19 20:35:45 2022] GET /identity/v3/endpoints?service_id=454c2183fa6041e291d324070e47d2f6&interface=public®ion_id=RegionOne => generated 194 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:45.433372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88603c7-6501-42e0-a213-37f1082334b9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:45.542327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f132b-f0fc-4fcd-8c24-dc24bb1b7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:45.543229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f132b-f0fc-4fcd-8c24-dc24bb1b7c8f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:45.543538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f132b-f0fc-4fcd-8c24-dc24bb1b7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:45.543819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f132b-f0fc-4fcd-8c24-dc24bb1b7c8f None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:45.544555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f132b-f0fc-4fcd-8c24-dc24bb1b7c8f None admin] RBAC: Adding query filter params (type=network) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:45.544875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f132b-f0fc-4fcd-8c24-dc24bb1b7c8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:45.546314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f132b-f0fc-4fcd-8c24-dc24bb1b7c8f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:45.560866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 409/777] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:35:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:45.569146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f132b-f0fc-4fcd-8c24-dc24bb1b7c8f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:45.602168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88603c7-6501-42e0-a213-37f1082334b9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:45.644808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b4f95-9403-4584-b429-8868eea64c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:45.645905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b4f95-9403-4584-b429-8868eea64c4c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:45.647044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b4f95-9403-4584-b429-8868eea64c4c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:45.647680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b4f95-9403-4584-b429-8868eea64c4c None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:45.648592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b4f95-9403-4584-b429-8868eea64c4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:45.650452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b4f95-9403-4584-b429-8868eea64c4c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:45.664472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 369/778] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jul 19 20:35:45 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:45.720389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:45.787276 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f132b-f0fc-4fcd-8c24-dc24bb1b7c8f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:45.849528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331603f8-e948-4355-893e-47a176656b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:45.850246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331603f8-e948-4355-893e-47a176656b91 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:45.850590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331603f8-e948-4355-893e-47a176656b91 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:45.850871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331603f8-e948-4355-893e-47a176656b91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:45.851739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331603f8-e948-4355-893e-47a176656b91 None admin] RBAC: Adding query filter params (interface=public, service_id=360d4aba16b24b12bbe1000ab9000747, region_id=RegionOne) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:45.852061 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331603f8-e948-4355-893e-47a176656b91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:45.853604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331603f8-e948-4355-893e-47a176656b91 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:45.869162 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 410/779] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jul 19 20:35:45 2022] GET /identity/v3/endpoints?service_id=360d4aba16b24b12bbe1000ab9000747&interface=public®ion_id=RegionOne => generated 194 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:45.878732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b4f95-9403-4584-b429-8868eea64c4c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:45.943099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171ab39-78f3-4e62-83a3-2b482a0f8938 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:45.943961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171ab39-78f3-4e62-83a3-2b482a0f8938 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:45.944897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171ab39-78f3-4e62-83a3-2b482a0f8938 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:45.949864 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:45.950703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 411/780] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:45.956762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a251d-6bdc-4c2d-805b-5b9f0eca5d94 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:45.957465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a251d-6bdc-4c2d-805b-5b9f0eca5d94 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:45.957945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a251d-6bdc-4c2d-805b-5b9f0eca5d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:45.978490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-255a251d-6bdc-4c2d-805b-5b9f0eca5d94 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:45.980186 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-255a251d-6bdc-4c2d-805b-5b9f0eca5d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:46.030529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b4f95-9403-4584-b429-8868eea64c4c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:46.088663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22b1a8-8a01-488a-a2e8-fea90001ee75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:46.090130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22b1a8-8a01-488a-a2e8-fea90001ee75 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:46.090626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22b1a8-8a01-488a-a2e8-fea90001ee75 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:46.091102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22b1a8-8a01-488a-a2e8-fea90001ee75 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:46.092224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22b1a8-8a01-488a-a2e8-fea90001ee75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:46.101679 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22b1a8-8a01-488a-a2e8-fea90001ee75 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:46.118369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 370/781] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jul 19 20:35:45 2022] GET /identity/v3/services => generated 772 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:46.129542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 412/782] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:35:46.161315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22b1a8-8a01-488a-a2e8-fea90001ee75 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:46.371069 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22b1a8-8a01-488a-a2e8-fea90001ee75 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:46.388847 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:46.423350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d36dd-89cc-46e5-a9a1-b680dd7c3da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:46.424881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d36dd-89cc-46e5-a9a1-b680dd7c3da9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:46.425473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d36dd-89cc-46e5-a9a1-b680dd7c3da9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:46.425960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d36dd-89cc-46e5-a9a1-b680dd7c3da9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:46.439379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d36dd-89cc-46e5-a9a1-b680dd7c3da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-3f0d36dd-89cc-46e5-a9a1-b680dd7c3da9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:46.448373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:46.449729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:46.450557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:46.453425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 371/783] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:46.467280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a251d-6bdc-4c2d-805b-5b9f0eca5d94 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:46.558527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:46.636233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a251d-6bdc-4c2d-805b-5b9f0eca5d94 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:46.683212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdf191-8789-4131-83ea-6f383c3ca306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:46.684275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdf191-8789-4131-83ea-6f383c3ca306 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:46.684789 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdf191-8789-4131-83ea-6f383c3ca306 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:46.685268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdf191-8789-4131-83ea-6f383c3ca306 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:46.686268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdf191-8789-4131-83ea-6f383c3ca306 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:46.686778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdf191-8789-4131-83ea-6f383c3ca306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:46.688475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdf191-8789-4131-83ea-6f383c3ca306 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:46.698641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 413/784] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:46.707527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d36dd-89cc-46e5-a9a1-b680dd7c3da9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:46.912741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d36dd-89cc-46e5-a9a1-b680dd7c3da9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:46.935578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86d9af-efb3-409f-9743-c955d42f0231 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:46.936076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86d9af-efb3-409f-9743-c955d42f0231 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:46.936560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86d9af-efb3-409f-9743-c955d42f0231 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:46.939239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 414/785] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:46.947338 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5ecf3-7eb5-4f89-a49b-7cf76d42460f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:46.947941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5ecf3-7eb5-4f89-a49b-7cf76d42460f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:46.948240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5ecf3-7eb5-4f89-a49b-7cf76d42460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:46.997956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e0b5ecf3-7eb5-4f89-a49b-7cf76d42460f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:46.999981 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e0b5ecf3-7eb5-4f89-a49b-7cf76d42460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:47.005514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f0197-3c14-4dc7-8649-cbc9d1f8fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:47.006350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0197-3c14-4dc7-8649-cbc9d1f8fbfd None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:47.006640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0197-3c14-4dc7-8649-cbc9d1f8fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:47.006922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0197-3c14-4dc7-8649-cbc9d1f8fbfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:47.043282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f0197-3c14-4dc7-8649-cbc9d1f8fbfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:47.045143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f0197-3c14-4dc7-8649-cbc9d1f8fbfd None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-fb6f0197-3c14-4dc7-8649-cbc9d1f8fbfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:47.090934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:47.092002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:47.093006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:47.093006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:47.093006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:47.093006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:47.095412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 372/786] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:47.120129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f0197-3c14-4dc7-8649-cbc9d1f8fbfd None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:47.160769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 415/787] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:35:47.174204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:47.212510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5ecf3-7eb5-4f89-a49b-7cf76d42460f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:47.274131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f0197-3c14-4dc7-8649-cbc9d1f8fbfd None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:47.280159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:47.321187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1314d86-bc4a-4383-96c3-e27b6e4d16b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:47.322284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1314d86-bc4a-4383-96c3-e27b6e4d16b1 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:47.322810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1314d86-bc4a-4383-96c3-e27b6e4d16b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:47.323352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1314d86-bc4a-4383-96c3-e27b6e4d16b1 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:47.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1314d86-bc4a-4383-96c3-e27b6e4d16b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:47.325403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1314d86-bc4a-4383-96c3-e27b6e4d16b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:47.327142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1314d86-bc4a-4383-96c3-e27b6e4d16b1 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:47.341935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 373/788] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:47.353435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1314d86-bc4a-4383-96c3-e27b6e4d16b1 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:47.374706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5ecf3-7eb5-4f89-a49b-7cf76d42460f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:47.417708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5287a6-8444-4876-b1d3-cbbf93ebee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:47.419281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5287a6-8444-4876-b1d3-cbbf93ebee11 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:47.419281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5287a6-8444-4876-b1d3-cbbf93ebee11 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:47.419281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5287a6-8444-4876-b1d3-cbbf93ebee11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:47.434443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5287a6-8444-4876-b1d3-cbbf93ebee11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-0c5287a6-8444-4876-b1d3-cbbf93ebee11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:47.446003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:47.447058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:47.447906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:47.448614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 416/789] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:47.449950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936f657-b899-4c7f-8296-33279386a95b None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:47.450243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936f657-b899-4c7f-8296-33279386a95b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:47.450521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936f657-b899-4c7f-8296-33279386a95b None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:47.451929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 417/790] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:47.457291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936f657-b899-4c7f-8296-33279386a95b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:47.471041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:47.499198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1314d86-bc4a-4383-96c3-e27b6e4d16b1 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:47.536889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd6d5b-2c29-41f5-baea-b669a5356b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:47.538135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd6d5b-2c29-41f5-baea-b669a5356b0b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:47.538635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd6d5b-2c29-41f5-baea-b669a5356b0b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:47.539245 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd6d5b-2c29-41f5-baea-b669a5356b0b None admin] PATH_INFO: `/v3/users/swift` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:47.548391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd6d5b-2c29-41f5-baea-b669a5356b0b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:47.550440 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd6d5b-2c29-41f5-baea-b669a5356b0b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-e7dd6d5b-2c29-41f5-baea-b669a5356b0b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:47.567694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:47.568718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:47.569725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:47.569725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:47.569725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:47.569725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:47.569725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:47.569725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:47.569725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:47.569725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:47.569725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:47.569725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:47.572471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 374/791] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:47.573982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc50f77-c4e6-4359-8fb1-3e0a28ecd8ea None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:47.574748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc50f77-c4e6-4359-8fb1-3e0a28ecd8ea None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:47.575231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc50f77-c4e6-4359-8fb1-3e0a28ecd8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:47.579915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936f657-b899-4c7f-8296-33279386a95b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:47.595043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-1fc50f77-c4e6-4359-8fb1-3e0a28ecd8ea None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:47.595703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:47.597941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1fc50f77-c4e6-4359-8fb1-3e0a28ecd8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:47.619481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c6608-c059-4a6c-8e9c-bc225afd853b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:47.620124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c6608-c059-4a6c-8e9c-bc225afd853b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:47.620452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c6608-c059-4a6c-8e9c-bc225afd853b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:47.620724 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c6608-c059-4a6c-8e9c-bc225afd853b None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:47.621511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c6608-c059-4a6c-8e9c-bc225afd853b None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:47.621820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c6608-c059-4a6c-8e9c-bc225afd853b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:47.623178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c6608-c059-4a6c-8e9c-bc225afd853b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:47.633711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 418/792] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:47.634726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c6608-c059-4a6c-8e9c-bc225afd853b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:47.678668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 375/793] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:35:47.679914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc50f77-c4e6-4359-8fb1-3e0a28ecd8ea None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:47.756737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:47.776295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c6608-c059-4a6c-8e9c-bc225afd853b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:47.795230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc50f77-c4e6-4359-8fb1-3e0a28ecd8ea None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:47.816788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d3f55-8820-4d1a-b8fb-b3e02700df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:47.817884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d3f55-8820-4d1a-b8fb-b3e02700df61 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:47.818364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d3f55-8820-4d1a-b8fb-b3e02700df61 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:47.818837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d3f55-8820-4d1a-b8fb-b3e02700df61 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:47.819898 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d3f55-8820-4d1a-b8fb-b3e02700df61 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:47.820457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d3f55-8820-4d1a-b8fb-b3e02700df61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:47.822167 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d3f55-8820-4d1a-b8fb-b3e02700df61 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:47.837156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f63b0-077d-4663-b3fc-bc42d986aa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:47.838267 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f63b0-077d-4663-b3fc-bc42d986aa11 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:47.838618 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f63b0-077d-4663-b3fc-bc42d986aa11 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:47.839612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f63b0-077d-4663-b3fc-bc42d986aa11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:47.848742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 419/794] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jul 19 20:35:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:47.849822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d3f55-8820-4d1a-b8fb-b3e02700df61 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:47.851850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f63b0-077d-4663-b3fc-bc42d986aa11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:47.853781 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f63b0-077d-4663-b3fc-bc42d986aa11 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-ef4f63b0-077d-4663-b3fc-bc42d986aa11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:47.875082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:47.876060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:47.877527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:47.877527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:47.877527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:47.877527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:47.877527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 376/795] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:47.878435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f63b0-077d-4663-b3fc-bc42d986aa11 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:47.892013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:47.972570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d3f55-8820-4d1a-b8fb-b3e02700df61 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:48.000917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f63b0-077d-4663-b3fc-bc42d986aa11 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:48.014463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f3b21-0855-44d6-b7b6-c0b89b314cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:48.015191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f3b21-0855-44d6-b7b6-c0b89b314cf6 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.015520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f3b21-0855-44d6-b7b6-c0b89b314cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.015809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f3b21-0855-44d6-b7b6-c0b89b314cf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.025659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:48.029681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f3b21-0855-44d6-b7b6-c0b89b314cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:48.043270 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20efb9-156a-4376-9e84-6ae6d028908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:48.044193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20efb9-156a-4376-9e84-6ae6d028908e None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.044376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20efb9-156a-4376-9e84-6ae6d028908e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.044632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20efb9-156a-4376-9e84-6ae6d028908e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-867f3b21-0855-44d6-b7b6-c0b89b314cf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:48.046764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:48.048581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:48.050589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:48.054695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 420/796] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:48.056432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f3b21-0855-44d6-b7b6-c0b89b314cf6 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:48.061780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20efb9-156a-4376-9e84-6ae6d028908e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:48.063682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20efb9-156a-4376-9e84-6ae6d028908e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-6c20efb9-156a-4376-9e84-6ae6d028908e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:48.086396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:48.088114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:48.090521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:48.090521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:48.090521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:48.090521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:48.090521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 377/797] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:48.090521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20efb9-156a-4376-9e84-6ae6d028908e None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:48.192101 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:48.200286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f3b21-0855-44d6-b7b6-c0b89b314cf6 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:48.207181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20efb9-156a-4376-9e84-6ae6d028908e None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:48.242597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b1da2-5a86-4425-9ed8-f74044b4ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:48.243609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b1da2-5a86-4425-9ed8-f74044b4ac4a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.244182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b1da2-5a86-4425-9ed8-f74044b4ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.244735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b1da2-5a86-4425-9ed8-f74044b4ac4a None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.245823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b1da2-5a86-4425-9ed8-f74044b4ac4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:48.246337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b1da2-5a86-4425-9ed8-f74044b4ac4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:48.247265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026eaf70-8d8f-43b3-9dd4-e46198abf747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:48.247940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eaf70-8d8f-43b3-9dd4-e46198abf747 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.248226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eaf70-8d8f-43b3-9dd4-e46198abf747 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.248505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eaf70-8d8f-43b3-9dd4-e46198abf747 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.248805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b1da2-5a86-4425-9ed8-f74044b4ac4a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:48.249265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026eaf70-8d8f-43b3-9dd4-e46198abf747 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:48.249624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026eaf70-8d8f-43b3-9dd4-e46198abf747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:48.251330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026eaf70-8d8f-43b3-9dd4-e46198abf747 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:48.267868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 378/799] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:48.267868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026eaf70-8d8f-43b3-9dd4-e46198abf747 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:48.268241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 421/799] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:48.269381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b1da2-5a86-4425-9ed8-f74044b4ac4a None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:48.306833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:48.398709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b1da2-5a86-4425-9ed8-f74044b4ac4a None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:48.406749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026eaf70-8d8f-43b3-9dd4-e46198abf747 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:48.444107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92696c8-695e-447e-966c-f35a668be7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:48.444836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92696c8-695e-447e-966c-f35a668be7ed None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.445129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92696c8-695e-447e-966c-f35a668be7ed None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.445452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92696c8-695e-447e-966c-f35a668be7ed None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.446395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92696c8-695e-447e-966c-f35a668be7ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:48.446708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92696c8-695e-447e-966c-f35a668be7ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:48.447836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadbace-34bd-4fd8-86f8-366d1948f928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:48.448106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92696c8-695e-447e-966c-f35a668be7ed None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:48.449387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadbace-34bd-4fd8-86f8-366d1948f928 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.450222 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadbace-34bd-4fd8-86f8-366d1948f928 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.450725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadbace-34bd-4fd8-86f8-366d1948f928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.462622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 379/800] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:48.463604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92696c8-695e-447e-966c-f35a668be7ed None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:48.465893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadbace-34bd-4fd8-86f8-366d1948f928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:48.469067 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadbace-34bd-4fd8-86f8-366d1948f928 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-aeadbace-34bd-4fd8-86f8-366d1948f928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:48.491878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:48.492822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:48.493728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:48.493728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:48.493728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:48.493728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:48.497983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 422/801] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:48.498652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadbace-34bd-4fd8-86f8-366d1948f928 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:48.508370 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:48.596358 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:48.605000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92696c8-695e-447e-966c-f35a668be7ed None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:48.625639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadbace-34bd-4fd8-86f8-366d1948f928 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:48.648975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a706e82-6657-4afc-af56-1f35bdd7f53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:48.649834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a706e82-6657-4afc-af56-1f35bdd7f53c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.650127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a706e82-6657-4afc-af56-1f35bdd7f53c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.650402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a706e82-6657-4afc-af56-1f35bdd7f53c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.658950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a706e82-6657-4afc-af56-1f35bdd7f53c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:48.660406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a706e82-6657-4afc-af56-1f35bdd7f53c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:48.667736 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df4577-6943-4fee-8958-afa347ddb968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:48.668537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df4577-6943-4fee-8958-afa347ddb968 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.668818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df4577-6943-4fee-8958-afa347ddb968 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.669093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df4577-6943-4fee-8958-afa347ddb968 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-1a706e82-6657-4afc-af56-1f35bdd7f53c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:48.676232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:48.677279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:48.678296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:48.678296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:48.678296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:48.678296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:48.678296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:48.678296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:48.678296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:48.678296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:48.678296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:35:48.678296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:48.678864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df4577-6943-4fee-8958-afa347ddb968 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:48.678864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df4577-6943-4fee-8958-afa347ddb968 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:48.681684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 380/802] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jul 19 20:35:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:48.683271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a706e82-6657-4afc-af56-1f35bdd7f53c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-59df4577-6943-4fee-8958-afa347ddb968 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:48.696005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:48.696994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:48.703983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:48.703983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:48.703983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:48.703983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:48.703983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:48.703983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:48.703983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:48.703983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:48.703983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 423/803] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:48.703983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7166f-2d4a-473e-a1fd-8bd3e1fd4445 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.703983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7166f-2d4a-473e-a1fd-8bd3e1fd4445 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.705083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7166f-2d4a-473e-a1fd-8bd3e1fd4445 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.705083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 424/804] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:48.705083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7166f-2d4a-473e-a1fd-8bd3e1fd4445 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:48.783650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:48.803407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a706e82-6657-4afc-af56-1f35bdd7f53c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:48.822076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7166f-2d4a-473e-a1fd-8bd3e1fd4445 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:48.838413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455826f1-8dc2-4492-9da7-bcdf2eae39da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:48.839670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455826f1-8dc2-4492-9da7-bcdf2eae39da None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.839960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455826f1-8dc2-4492-9da7-bcdf2eae39da None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.840239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455826f1-8dc2-4492-9da7-bcdf2eae39da None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.840971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455826f1-8dc2-4492-9da7-bcdf2eae39da None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:48.841284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455826f1-8dc2-4492-9da7-bcdf2eae39da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:48.842695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455826f1-8dc2-4492-9da7-bcdf2eae39da None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:48.858913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 381/805] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:48.858913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3de90-b9eb-466b-8e19-97b7c728d05c None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.858913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3de90-b9eb-466b-8e19-97b7c728d05c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.858913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3de90-b9eb-466b-8e19-97b7c728d05c None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.858913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 382/806] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:48.859362 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3de90-b9eb-466b-8e19-97b7c728d05c None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:48.866248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6037bf-5606-4b31-ac34-5b301ad4dd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:48.866934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6037bf-5606-4b31-ac34-5b301ad4dd25 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.867218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6037bf-5606-4b31-ac34-5b301ad4dd25 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.867491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6037bf-5606-4b31-ac34-5b301ad4dd25 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.868340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6037bf-5606-4b31-ac34-5b301ad4dd25 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:48.868648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6037bf-5606-4b31-ac34-5b301ad4dd25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:48.870685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6037bf-5606-4b31-ac34-5b301ad4dd25 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:48.881791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:48.892680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 425/807] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jul 19 20:35:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:48.894039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2baed-acd9-4daa-94f7-0be616b74221 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:48.894330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2baed-acd9-4daa-94f7-0be616b74221 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:48.894607 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2baed-acd9-4daa-94f7-0be616b74221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:48.911134 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-5aa2baed-acd9-4daa-94f7-0be616b74221 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:48.912346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5aa2baed-acd9-4daa-94f7-0be616b74221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:48.973897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3de90-b9eb-466b-8e19-97b7c728d05c None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:48.982514 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 426/808] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:35:48.983939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2baed-acd9-4daa-94f7-0be616b74221 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:49.012102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3747b59-da09-4962-a245-f08e1450a857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:49.012772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3747b59-da09-4962-a245-f08e1450a857 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:49.013060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3747b59-da09-4962-a245-f08e1450a857 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:49.013340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3747b59-da09-4962-a245-f08e1450a857 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:49.014166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3747b59-da09-4962-a245-f08e1450a857 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:49.014476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3747b59-da09-4962-a245-f08e1450a857 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:49.015910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3747b59-da09-4962-a245-f08e1450a857 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:49.028237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 383/809] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:49.029755 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828b149-d66c-40cc-8e63-a93d399f0b20 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:49.030053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828b149-d66c-40cc-8e63-a93d399f0b20 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:49.030334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828b149-d66c-40cc-8e63-a93d399f0b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:49.047373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-6828b149-d66c-40cc-8e63-a93d399f0b20 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:49.048594 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6828b149-d66c-40cc-8e63-a93d399f0b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:49.055244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:49.102885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2baed-acd9-4daa-94f7-0be616b74221 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:49.119788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 384/810] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:35:49.121368 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828b149-d66c-40cc-8e63-a93d399f0b20 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:49.126624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:49.142699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bd1ac-6ce3-4614-8cb8-0ac1752f7ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:49.143515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd1ac-6ce3-4614-8cb8-0ac1752f7ed0 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:49.144108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd1ac-6ce3-4614-8cb8-0ac1752f7ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:49.144108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd1ac-6ce3-4614-8cb8-0ac1752f7ed0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:49.151935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bd1ac-6ce3-4614-8cb8-0ac1752f7ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:49.153620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bd1ac-6ce3-4614-8cb8-0ac1752f7ed0 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-406bd1ac-6ce3-4614-8cb8-0ac1752f7ed0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:49.174556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:49.175677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:49.176612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:49.176612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:49.176612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:49.176612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:49.176612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:49.176612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:49.176612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:49.176612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:49.176612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:35:49.176612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:49.176612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 427/811] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jul 19 20:35:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:49.177213 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bd1ac-6ce3-4614-8cb8-0ac1752f7ed0 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:49.240102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828b149-d66c-40cc-8e63-a93d399f0b20 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:49.276149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e3958-fa61-4138-80f9-757a97c58256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:49.276982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e3958-fa61-4138-80f9-757a97c58256 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:49.277262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e3958-fa61-4138-80f9-757a97c58256 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:49.277560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e3958-fa61-4138-80f9-757a97c58256 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:49.288367 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e3958-fa61-4138-80f9-757a97c58256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:49.289865 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e3958-fa61-4138-80f9-757a97c58256 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:49.296192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bd1ac-6ce3-4614-8cb8-0ac1752f7ed0 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-a14e3958-fa61-4138-80f9-757a97c58256 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:49.308842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:49.309967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:49.311883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:49.311883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:49.311883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:49.311883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:49.311883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 385/812] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:49.311883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e3958-fa61-4138-80f9-757a97c58256 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:49.315292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:49.334058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a96652-85f5-40d0-8205-cfc19e192bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:49.334870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a96652-85f5-40d0-8205-cfc19e192bc7 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:49.335157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a96652-85f5-40d0-8205-cfc19e192bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:49.335433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a96652-85f5-40d0-8205-cfc19e192bc7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:49.336088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a96652-85f5-40d0-8205-cfc19e192bc7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:49.337479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a96652-85f5-40d0-8205-cfc19e192bc7 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-b4a96652-85f5-40d0-8205-cfc19e192bc7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:49.349779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:49.351042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:49.352328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 428/813] 10.222.0.70 () {56 vars in 1129 bytes} [Tue Jul 19 20:35:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:49.353177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a96652-85f5-40d0-8205-cfc19e192bc7 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:49.396620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:49.452803 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e3958-fa61-4138-80f9-757a97c58256 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:49.475635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a96652-85f5-40d0-8205-cfc19e192bc7 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:49.516164 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d34a9-7ae7-4019-9c3e-a6ec9f50bc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:49.517081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d34a9-7ae7-4019-9c3e-a6ec9f50bc28 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:49.517567 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d34a9-7ae7-4019-9c3e-a6ec9f50bc28 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:49.517916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d34a9-7ae7-4019-9c3e-a6ec9f50bc28 None admin] PATH_INFO: `/v3/services/network` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:49.518716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d34a9-7ae7-4019-9c3e-a6ec9f50bc28 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:49.520226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d34a9-7ae7-4019-9c3e-a6ec9f50bc28 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-b70d34a9-7ae7-4019-9c3e-a6ec9f50bc28 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:49.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:49.535704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:49.536761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 429/814] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:49 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:49.538082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d34a9-7ae7-4019-9c3e-a6ec9f50bc28 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:49.591708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:49.611339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebe3eb-82af-42d7-a0a9-fb4911fbf1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:49.612244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebe3eb-82af-42d7-a0a9-fb4911fbf1d6 None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:49.612539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebe3eb-82af-42d7-a0a9-fb4911fbf1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:49.612808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebe3eb-82af-42d7-a0a9-fb4911fbf1d6 None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/334a380fa68240c79a72746ee4a5f9dc/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:49.614861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebe3eb-82af-42d7-a0a9-fb4911fbf1d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=334a380fa68240c79a72746ee4a5f9dc, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:49.616296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebe3eb-82af-42d7-a0a9-fb4911fbf1d6 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:49.631966 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 386/815] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:35:49 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/334a380fa68240c79a72746ee4a5f9dc/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:35:49.632729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebe3eb-82af-42d7-a0a9-fb4911fbf1d6 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:49.660745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d34a9-7ae7-4019-9c3e-a6ec9f50bc28 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:49.713472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49342ca9-5a13-44e6-9d64-4d4ebe243999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:49.715042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342ca9-5a13-44e6-9d64-4d4ebe243999 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:49.715473 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342ca9-5a13-44e6-9d64-4d4ebe243999 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:49.715948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342ca9-5a13-44e6-9d64-4d4ebe243999 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:49.717026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49342ca9-5a13-44e6-9d64-4d4ebe243999 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:49.717523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49342ca9-5a13-44e6-9d64-4d4ebe243999 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:49.719142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49342ca9-5a13-44e6-9d64-4d4ebe243999 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:49.745565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 430/816] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jul 19 20:35:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:49.745565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49342ca9-5a13-44e6-9d64-4d4ebe243999 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:49.775743 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebe3eb-82af-42d7-a0a9-fb4911fbf1d6 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:49.798453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:49.824454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed0332-76db-44c0-89bc-d34f4b576263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:49.825128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed0332-76db-44c0-89bc-d34f4b576263 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:49.825466 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed0332-76db-44c0-89bc-d34f4b576263 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:49.825752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed0332-76db-44c0-89bc-d34f4b576263 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:49.826764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed0332-76db-44c0-89bc-d34f4b576263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:49.827079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed0332-76db-44c0-89bc-d34f4b576263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:49.828463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed0332-76db-44c0-89bc-d34f4b576263 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:49.842015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 387/817] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:49.842703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed0332-76db-44c0-89bc-d34f4b576263 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:49.864560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:49.868801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49342ca9-5a13-44e6-9d64-4d4ebe243999 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:49.916829 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34042496-7be2-49eb-907f-de97efef0a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:49.918270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34042496-7be2-49eb-907f-de97efef0a48 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:49.918556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34042496-7be2-49eb-907f-de97efef0a48 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:49.918894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34042496-7be2-49eb-907f-de97efef0a48 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:49.919775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34042496-7be2-49eb-907f-de97efef0a48 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:49.920107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34042496-7be2-49eb-907f-de97efef0a48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:49.921727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34042496-7be2-49eb-907f-de97efef0a48 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:49.937402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 431/818] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jul 19 20:35:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:49.937402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34042496-7be2-49eb-907f-de97efef0a48 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:50.006851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed0332-76db-44c0-89bc-d34f4b576263 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:50.061532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4d44a-a27f-496e-ac49-8a328896a471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:50.063058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4d44a-a27f-496e-ac49-8a328896a471 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:50.063622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4d44a-a27f-496e-ac49-8a328896a471 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:50.064148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4d44a-a27f-496e-ac49-8a328896a471 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:50.065350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4d44a-a27f-496e-ac49-8a328896a471 None admin] RBAC: Adding query filter params (name=network) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:50.065895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4d44a-a27f-496e-ac49-8a328896a471 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:50.067699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4d44a-a27f-496e-ac49-8a328896a471 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:50.083033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 388/819] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:35:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:50.083033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4d44a-a27f-496e-ac49-8a328896a471 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:50.096719 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34042496-7be2-49eb-907f-de97efef0a48 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:50.129910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:50.135261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12126c-8d2f-4e03-8063-33bd03b9473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:50.136858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12126c-8d2f-4e03-8063-33bd03b9473b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:50.137253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12126c-8d2f-4e03-8063-33bd03b9473b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:50.137529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12126c-8d2f-4e03-8063-33bd03b9473b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:50.174344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:50.206890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4d44a-a27f-496e-ac49-8a328896a471 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:50.221941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12126c-8d2f-4e03-8063-33bd03b9473b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:50.223429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12126c-8d2f-4e03-8063-33bd03b9473b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:50.246988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330b92-6426-42e7-94e3-5c7433c11b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:50.247787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330b92-6426-42e7-94e3-5c7433c11b76 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:50.248073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330b92-6426-42e7-94e3-5c7433c11b76 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:50.248350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330b92-6426-42e7-94e3-5c7433c11b76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:50.255887 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c330b92-6426-42e7-94e3-5c7433c11b76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:50.257344 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c330b92-6426-42e7-94e3-5c7433c11b76 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-1c330b92-6426-42e7-94e3-5c7433c11b76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:50.272339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:50.273286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:50.274257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:50.274257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:50.274257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:50.274257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:50.274257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:50.274257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:50.274257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:50.274257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:50.274257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 389/820] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:50.274745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330b92-6426-42e7-94e3-5c7433c11b76 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:50.343839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 432/821] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:35:50.349860 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12126c-8d2f-4e03-8063-33bd03b9473b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:50.405070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:50.408614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330b92-6426-42e7-94e3-5c7433c11b76 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:50.419526 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:50.450035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068bc28-ffc9-4c69-a8c8-1e209845c08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:50.450866 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068bc28-ffc9-4c69-a8c8-1e209845c08d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:50.450997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068bc28-ffc9-4c69-a8c8-1e209845c08d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:50.451250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068bc28-ffc9-4c69-a8c8-1e209845c08d None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:50.451965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068bc28-ffc9-4c69-a8c8-1e209845c08d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:50.452272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068bc28-ffc9-4c69-a8c8-1e209845c08d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:50.453671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068bc28-ffc9-4c69-a8c8-1e209845c08d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:50.463528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 390/822] 10.222.0.70 () {56 vars in 1124 bytes} [Tue Jul 19 20:35:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:50.463528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068bc28-ffc9-4c69-a8c8-1e209845c08d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:50.471137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12126c-8d2f-4e03-8063-33bd03b9473b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:50.514021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf57bc-3e61-4036-a32d-0b0fc56b902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:50.514910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf57bc-3e61-4036-a32d-0b0fc56b902c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:50.515286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf57bc-3e61-4036-a32d-0b0fc56b902c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:50.515661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf57bc-3e61-4036-a32d-0b0fc56b902c None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:50.516471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf57bc-3e61-4036-a32d-0b0fc56b902c None admin] RBAC: Adding query filter params (type=network) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:50.516875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf57bc-3e61-4036-a32d-0b0fc56b902c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:50.518445 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf57bc-3e61-4036-a32d-0b0fc56b902c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:50.523681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 433/823] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:35:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:50.524579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf57bc-3e61-4036-a32d-0b0fc56b902c None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:50.572403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068bc28-ffc9-4c69-a8c8-1e209845c08d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:50.609503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac07cc4-da89-412f-a4c0-29a155e1b627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:50.610341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac07cc4-da89-412f-a4c0-29a155e1b627 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:50.610724 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac07cc4-da89-412f-a4c0-29a155e1b627 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:50.611098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac07cc4-da89-412f-a4c0-29a155e1b627 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:50.611997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac07cc4-da89-412f-a4c0-29a155e1b627 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:50.612406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac07cc4-da89-412f-a4c0-29a155e1b627 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:50.613979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac07cc4-da89-412f-a4c0-29a155e1b627 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:50.627732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 391/824] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:50.633310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac07cc4-da89-412f-a4c0-29a155e1b627 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:50.640655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf57bc-3e61-4036-a32d-0b0fc56b902c None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:50.662174 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:50.672606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:50.688098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7676bc-ba34-4e99-886f-95ea8bcf2cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:50.689133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7676bc-ba34-4e99-886f-95ea8bcf2cd1 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:50.689606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7676bc-ba34-4e99-886f-95ea8bcf2cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:50.689988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7676bc-ba34-4e99-886f-95ea8bcf2cd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:50.702739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7676bc-ba34-4e99-886f-95ea8bcf2cd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:50.704612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7676bc-ba34-4e99-886f-95ea8bcf2cd1 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-ef7676bc-ba34-4e99-886f-95ea8bcf2cd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:50.724182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:50.725159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:50.730907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:50.730907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:50.730907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:50.730907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:50.730907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 434/825] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:50.730907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7676bc-ba34-4e99-886f-95ea8bcf2cd1 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:50.760475 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac07cc4-da89-412f-a4c0-29a155e1b627 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:50.802399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e44f9-a96c-4c45-9abe-9b85d3865c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:50.803066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e44f9-a96c-4c45-9abe-9b85d3865c67 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:50.803351 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e44f9-a96c-4c45-9abe-9b85d3865c67 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:50.803630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e44f9-a96c-4c45-9abe-9b85d3865c67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:50.804380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e44f9-a96c-4c45-9abe-9b85d3865c67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:50.805801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e44f9-a96c-4c45-9abe-9b85d3865c67 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:50.841396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 392/826] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jul 19 20:35:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:50.841396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e44f9-a96c-4c45-9abe-9b85d3865c67 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:50.856955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7676bc-ba34-4e99-886f-95ea8bcf2cd1 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:50.918964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79aa702-0ac9-4ab3-b4e2-f70c36ba985e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:50.920077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aa702-0ac9-4ab3-b4e2-f70c36ba985e None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:50.920434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aa702-0ac9-4ab3-b4e2-f70c36ba985e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:50.920788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79aa702-0ac9-4ab3-b4e2-f70c36ba985e None admin] PATH_INFO: `/v3/endpoints` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:50.921835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79aa702-0ac9-4ab3-b4e2-f70c36ba985e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:50.924375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79aa702-0ac9-4ab3-b4e2-f70c36ba985e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:50.951715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:50.969220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 435/827] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jul 19 20:35:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:50.970571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79aa702-0ac9-4ab3-b4e2-f70c36ba985e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:50.989818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e44f9-a96c-4c45-9abe-9b85d3865c67 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:51.041853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:51.084780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce4a3a-ca7c-4f98-b459-b0b0af1a1ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:51.086022 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce4a3a-ca7c-4f98-b459-b0b0af1a1ce9 None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:51.086463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce4a3a-ca7c-4f98-b459-b0b0af1a1ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:51.086888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce4a3a-ca7c-4f98-b459-b0b0af1a1ce9 None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/2711df504d424fa0b05cd16d7fd5acfb/roles/2de43a7c794c4df181f6f4c58a83a9bf` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:51.105316 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce4a3a-ca7c-4f98-b459-b0b0af1a1ce9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=2711df504d424fa0b05cd16d7fd5acfb, role_id=2de43a7c794c4df181f6f4c58a83a9bf)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:51.107049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce4a3a-ca7c-4f98-b459-b0b0af1a1ce9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:51.131326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 393/828] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:35:50 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/2711df504d424fa0b05cd16d7fd5acfb/roles/2de43a7c794c4df181f6f4c58a83a9bf => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:35:51.155353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79aa702-0ac9-4ab3-b4e2-f70c36ba985e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:51.202122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902fdf6-6006-47b3-83c8-9eacb001a3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:51.202831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902fdf6-6006-47b3-83c8-9eacb001a3ca None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:51.203114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902fdf6-6006-47b3-83c8-9eacb001a3ca None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:51.203422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902fdf6-6006-47b3-83c8-9eacb001a3ca None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:51.204365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902fdf6-6006-47b3-83c8-9eacb001a3ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:51.204675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902fdf6-6006-47b3-83c8-9eacb001a3ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:51.206386 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902fdf6-6006-47b3-83c8-9eacb001a3ca None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:51.223478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 436/829] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:51.229075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce4a3a-ca7c-4f98-b459-b0b0af1a1ce9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:51.387158 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce4a3a-ca7c-4f98-b459-b0b0af1a1ce9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:51.418557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:51.432180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe287666-9ac5-4e83-a0c1-8bc1c48b14f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:51.433581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe287666-9ac5-4e83-a0c1-8bc1c48b14f7 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:51.433870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe287666-9ac5-4e83-a0c1-8bc1c48b14f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:51.434148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe287666-9ac5-4e83-a0c1-8bc1c48b14f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:51.441789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe287666-9ac5-4e83-a0c1-8bc1c48b14f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:51.443268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe287666-9ac5-4e83-a0c1-8bc1c48b14f7 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-fe287666-9ac5-4e83-a0c1-8bc1c48b14f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:51.459302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:51.460317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:51.461211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:51.461211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:51.461211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:51.461211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:51.461211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:51.461211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:51.461211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:51.461211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:51.461211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 394/830] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:51.470514 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902fdf6-6006-47b3-83c8-9eacb001a3ca None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:51.570580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:51.580710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902fdf6-6006-47b3-83c8-9eacb001a3ca None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:51.618888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a2914-5239-4e39-b8e0-4e1a8892ed99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:51.619568 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a2914-5239-4e39-b8e0-4e1a8892ed99 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:51.619850 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a2914-5239-4e39-b8e0-4e1a8892ed99 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:51.620314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a2914-5239-4e39-b8e0-4e1a8892ed99 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:51.621639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a2914-5239-4e39-b8e0-4e1a8892ed99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:51.621954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a2914-5239-4e39-b8e0-4e1a8892ed99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:51.623390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a2914-5239-4e39-b8e0-4e1a8892ed99 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:51.636612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 437/831] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:51.642782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe287666-9ac5-4e83-a0c1-8bc1c48b14f7 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:51.752952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe287666-9ac5-4e83-a0c1-8bc1c48b14f7 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:51.787706 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce6490-b1fe-48e3-847f-6c2e3d329085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:51.788383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce6490-b1fe-48e3-847f-6c2e3d329085 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:51.788670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce6490-b1fe-48e3-847f-6c2e3d329085 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:51.788950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce6490-b1fe-48e3-847f-6c2e3d329085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:51.789826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce6490-b1fe-48e3-847f-6c2e3d329085 None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=e1d09e168fc54f1da06b719989ae8ef0) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:51.790141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce6490-b1fe-48e3-847f-6c2e3d329085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:51.791573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce6490-b1fe-48e3-847f-6c2e3d329085 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:51.799068 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 395/832] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:51 2022] GET /identity/v3/role_assignments?user.id=e1d09e168fc54f1da06b719989ae8ef0&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=85420c68300642aabfd730fb8fef8347 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:35:51.889144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bd397-5fbc-41d9-a0af-4d83221af13a None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:51.889471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bd397-5fbc-41d9-a0af-4d83221af13a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:51.889707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bd397-5fbc-41d9-a0af-4d83221af13a None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:51.891519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 438/833] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:51.897336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c85a35-7e2d-4f5c-bc7f-c6abe3579b8b None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:51.897687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c85a35-7e2d-4f5c-bc7f-c6abe3579b8b None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:51.897935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c85a35-7e2d-4f5c-bc7f-c6abe3579b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:51.920610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-68c85a35-7e2d-4f5c-bc7f-c6abe3579b8b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:51.923259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-68c85a35-7e2d-4f5c-bc7f-c6abe3579b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:51.980823 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:52.024674 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 396/834] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:35:52.063835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bd397-5fbc-41d9-a0af-4d83221af13a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:52.186416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bd397-5fbc-41d9-a0af-4d83221af13a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:52.246092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad7ed7-401d-4a5d-a2d6-92e6955dd22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:52.247433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad7ed7-401d-4a5d-a2d6-92e6955dd22e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:52.247920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad7ed7-401d-4a5d-a2d6-92e6955dd22e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:52.248371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad7ed7-401d-4a5d-a2d6-92e6955dd22e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:52.269716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad7ed7-401d-4a5d-a2d6-92e6955dd22e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-4cad7ed7-401d-4a5d-a2d6-92e6955dd22e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:52.285197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:52.286218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:35:52.287040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:52.288172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 439/835] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:52.296991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c85a35-7e2d-4f5c-bc7f-c6abe3579b8b None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:52.425188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c85a35-7e2d-4f5c-bc7f-c6abe3579b8b None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:52.466410 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c6f60-41c8-47d0-b5d0-1c07442c141c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:52.467462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c6f60-41c8-47d0-b5d0-1c07442c141c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:52.467945 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c6f60-41c8-47d0-b5d0-1c07442c141c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:52.468412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c6f60-41c8-47d0-b5d0-1c07442c141c None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:52.469687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c6f60-41c8-47d0-b5d0-1c07442c141c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:52.470259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c6f60-41c8-47d0-b5d0-1c07442c141c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:52.471871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c6f60-41c8-47d0-b5d0-1c07442c141c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:52.483230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 397/836] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:35:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:52.489872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad7ed7-401d-4a5d-a2d6-92e6955dd22e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:52.628043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:52.750925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad7ed7-401d-4a5d-a2d6-92e6955dd22e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:52.794325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e099f6f-474f-40e7-b236-535c5e5445f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:52.795147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e099f6f-474f-40e7-b236-535c5e5445f1 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:52.795433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e099f6f-474f-40e7-b236-535c5e5445f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:52.795713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e099f6f-474f-40e7-b236-535c5e5445f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:52.808096 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e099f6f-474f-40e7-b236-535c5e5445f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:52.809584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e099f6f-474f-40e7-b236-535c5e5445f1 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-0e099f6f-474f-40e7-b236-535c5e5445f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:52.828846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:52.829936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:52.830989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:52.830989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:52.830989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:52.830989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:52.830989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 440/837] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:52.836593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c6f60-41c8-47d0-b5d0-1c07442c141c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:52.887236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:52.948911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c6f60-41c8-47d0-b5d0-1c07442c141c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:52.986849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe3e8e-146f-428b-9c5d-8b2c42a583ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:52.987538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe3e8e-146f-428b-9c5d-8b2c42a583ea None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:52.987918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe3e8e-146f-428b-9c5d-8b2c42a583ea None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:52.988124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe3e8e-146f-428b-9c5d-8b2c42a583ea None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:52.988830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe3e8e-146f-428b-9c5d-8b2c42a583ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:52.989137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe3e8e-146f-428b-9c5d-8b2c42a583ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:52.990585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe3e8e-146f-428b-9c5d-8b2c42a583ea None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:53.004737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 398/838] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:53.010366 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e099f6f-474f-40e7-b236-535c5e5445f1 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:53.128043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:53.185488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e099f6f-474f-40e7-b236-535c5e5445f1 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:53.225076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddaa1c4-71b3-4273-80a4-d06a71007bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:53.225940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaa1c4-71b3-4273-80a4-d06a71007bdf None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:53.226195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaa1c4-71b3-4273-80a4-d06a71007bdf None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:53.226474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaa1c4-71b3-4273-80a4-d06a71007bdf None admin] PATH_INFO: `/v3/users/swift` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:53.235615 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddaa1c4-71b3-4273-80a4-d06a71007bdf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:53.237090 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddaa1c4-71b3-4273-80a4-d06a71007bdf None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-9ddaa1c4-71b3-4273-80a4-d06a71007bdf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:53.253778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:53.255215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:53.256180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:53.256180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:53.256180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:53.256180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:53.256180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:53.256180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:53.256180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:53.256180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:53.256180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:35:53.256180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:53.256180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 441/839] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:35:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:53.264827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe3e8e-146f-428b-9c5d-8b2c42a583ea None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:53.416813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe3e8e-146f-428b-9c5d-8b2c42a583ea None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:53.438766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08759bd-f6bb-4ab6-ac17-2fc586f6fc7f None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:53.439605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08759bd-f6bb-4ab6-ac17-2fc586f6fc7f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:53.439892 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08759bd-f6bb-4ab6-ac17-2fc586f6fc7f None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:53.445521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 442/840] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:53.446393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:53.447810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5ecae-a3e7-49b5-bc1c-7100d7b1f9ae None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:53.448097 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5ecae-a3e7-49b5-bc1c-7100d7b1f9ae None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:53.448951 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5ecae-a3e7-49b5-bc1c-7100d7b1f9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:53.469073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-ced5ecae-a3e7-49b5-bc1c-7100d7b1f9ae None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:53.470354 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ced5ecae-a3e7-49b5-bc1c-7100d7b1f9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:53.474808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c676de-8e84-4bf7-90ac-06c5471b3dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:53.475493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c676de-8e84-4bf7-90ac-06c5471b3dd6 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:53.475779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c676de-8e84-4bf7-90ac-06c5471b3dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:53.476051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c676de-8e84-4bf7-90ac-06c5471b3dd6 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:53.476900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c676de-8e84-4bf7-90ac-06c5471b3dd6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:53.477208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c676de-8e84-4bf7-90ac-06c5471b3dd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:53.478861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c676de-8e84-4bf7-90ac-06c5471b3dd6 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:53.510019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 399/841] 10.222.0.70 () {56 vars in 1135 bytes} [Tue Jul 19 20:35:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:53.516892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c676de-8e84-4bf7-90ac-06c5471b3dd6 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:53.580380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 443/842] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:35:53.610753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5ecae-a3e7-49b5-bc1c-7100d7b1f9ae None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:53.676879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c676de-8e84-4bf7-90ac-06c5471b3dd6 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:53.703965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:53.718455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f425de-3237-4548-af7f-9f44e50ec06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:53.719124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f425de-3237-4548-af7f-9f44e50ec06e None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:53.719408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f425de-3237-4548-af7f-9f44e50ec06e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:53.719684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f425de-3237-4548-af7f-9f44e50ec06e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:53.752282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:53.765558 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5ecae-a3e7-49b5-bc1c-7100d7b1f9ae None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:53.797713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f425de-3237-4548-af7f-9f44e50ec06e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:53.799266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f425de-3237-4548-af7f-9f44e50ec06e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:53.804381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2929f-1e24-497d-a07a-25f4e137bf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:53.805500 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2929f-1e24-497d-a07a-25f4e137bf67 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:53.805989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2929f-1e24-497d-a07a-25f4e137bf67 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:53.806443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2929f-1e24-497d-a07a-25f4e137bf67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:53.827893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2929f-1e24-497d-a07a-25f4e137bf67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-0fc2929f-1e24-497d-a07a-25f4e137bf67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:53.844749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:53.846686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:35:53.848684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:53.860978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 444/843] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:35:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:53.862740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961ac8f-3b14-4ec1-80d5-b0a1790a7b43 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:53.863222 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961ac8f-3b14-4ec1-80d5-b0a1790a7b43 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:53.863509 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961ac8f-3b14-4ec1-80d5-b0a1790a7b43 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:53.869325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 445/844] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:53.869325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961ac8f-3b14-4ec1-80d5-b0a1790a7b43 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:53.921319 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 400/845] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:53 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 20:35:53.921319 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52ee0a-a043-4c4b-909b-f499495cc7f4 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:53.921319 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52ee0a-a043-4c4b-909b-f499495cc7f4 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:53.921319 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52ee0a-a043-4c4b-909b-f499495cc7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:53.941587 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-3d52ee0a-a043-4c4b-909b-f499495cc7f4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:53.943400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3d52ee0a-a043-4c4b-909b-f499495cc7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:54.010188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961ac8f-3b14-4ec1-80d5-b0a1790a7b43 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:54.021238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:54.026201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:54.035177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 401/846] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:35:54.036574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52ee0a-a043-4c4b-909b-f499495cc7f4 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:54.062981 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66735d04-6354-42be-9db2-d526b079a900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:54.064009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735d04-6354-42be-9db2-d526b079a900 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.064294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735d04-6354-42be-9db2-d526b079a900 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.064716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735d04-6354-42be-9db2-d526b079a900 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.065540 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66735d04-6354-42be-9db2-d526b079a900 None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:54.065854 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66735d04-6354-42be-9db2-d526b079a900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:54.067233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66735d04-6354-42be-9db2-d526b079a900 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:54.077006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 446/847] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:35:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:54.082587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66735d04-6354-42be-9db2-d526b079a900 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:54.163120 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52ee0a-a043-4c4b-909b-f499495cc7f4 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:54.201609 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c9de2-947a-4775-8dbc-345369a3fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:54.202416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c9de2-947a-4775-8dbc-345369a3fe6d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.202699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c9de2-947a-4775-8dbc-345369a3fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.202977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c9de2-947a-4775-8dbc-345369a3fe6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.208564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66735d04-6354-42be-9db2-d526b079a900 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:54.214710 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c9de2-947a-4775-8dbc-345369a3fe6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:54.216796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c9de2-947a-4775-8dbc-345369a3fe6d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-515c9de2-947a-4775-8dbc-345369a3fe6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:54.238243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:54.239402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:54.240913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:54.240913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:54.240913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:54.240913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:54.240913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 402/848] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:54.241574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c9de2-947a-4775-8dbc-345369a3fe6d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:54.247974 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58464f3-e999-4e5a-b2da-d544d90c243a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:54.248871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58464f3-e999-4e5a-b2da-d544d90c243a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.249228 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58464f3-e999-4e5a-b2da-d544d90c243a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.249544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58464f3-e999-4e5a-b2da-d544d90c243a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.250217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58464f3-e999-4e5a-b2da-d544d90c243a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:54.251592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58464f3-e999-4e5a-b2da-d544d90c243a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-a58464f3-e999-4e5a-b2da-d544d90c243a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:54.264796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:54.266441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:54.267774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 447/849] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:35:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:54.269133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bde00d-55d3-4bcf-a6ed-e023666c2b5b None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.269133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bde00d-55d3-4bcf-a6ed-e023666c2b5b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.269133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bde00d-55d3-4bcf-a6ed-e023666c2b5b None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.269559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 448/850] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:54.270213 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bde00d-55d3-4bcf-a6ed-e023666c2b5b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:54.297705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:54.316407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:54.358002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c9de2-947a-4775-8dbc-345369a3fe6d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:54.393115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bde00d-55d3-4bcf-a6ed-e023666c2b5b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:54.405198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f55f5-34ef-4206-bfae-b474a13c9a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:54.406059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f55f5-34ef-4206-bfae-b474a13c9a97 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.406347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f55f5-34ef-4206-bfae-b474a13c9a97 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.406619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f55f5-34ef-4206-bfae-b474a13c9a97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.417928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f55f5-34ef-4206-bfae-b474a13c9a97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:54.419488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f55f5-34ef-4206-bfae-b474a13c9a97 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:54.431030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd6128-ac10-4b1b-8a2d-6fc989aa12bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:54.431853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd6128-ac10-4b1b-8a2d-6fc989aa12bd None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.432226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd6128-ac10-4b1b-8a2d-6fc989aa12bd None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.433112 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd6128-ac10-4b1b-8a2d-6fc989aa12bd None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.433990 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd6128-ac10-4b1b-8a2d-6fc989aa12bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:54.434446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd6128-ac10-4b1b-8a2d-6fc989aa12bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:54.435941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd6128-ac10-4b1b-8a2d-6fc989aa12bd None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-4d9f55f5-34ef-4206-bfae-b474a13c9a97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:54.438317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:54.439429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:54.441521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:54.441521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:54.441521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:54.441521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:54.441521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 403/851] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:54.441521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f55f5-34ef-4206-bfae-b474a13c9a97 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:54.463605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 449/852] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:54.463605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1ffb6-7ca7-4e6d-b7fb-fce0b8b07692 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.463605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1ffb6-7ca7-4e6d-b7fb-fce0b8b07692 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.463605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1ffb6-7ca7-4e6d-b7fb-fce0b8b07692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.485571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-2fb1ffb6-7ca7-4e6d-b7fb-fce0b8b07692 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:54.486934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2fb1ffb6-7ca7-4e6d-b7fb-fce0b8b07692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:54.556213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:54.560695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 450/853] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:35:54.561590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1ffb6-7ca7-4e6d-b7fb-fce0b8b07692 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:54.566480 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f55f5-34ef-4206-bfae-b474a13c9a97 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:54.606818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:54.609560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56fcd7-fcb2-4560-9957-7593c9b1cba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:54.610297 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56fcd7-fcb2-4560-9957-7593c9b1cba7 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.610514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56fcd7-fcb2-4560-9957-7593c9b1cba7 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.610845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56fcd7-fcb2-4560-9957-7593c9b1cba7 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.611503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56fcd7-fcb2-4560-9957-7593c9b1cba7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:54.611819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56fcd7-fcb2-4560-9957-7593c9b1cba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:54.613180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56fcd7-fcb2-4560-9957-7593c9b1cba7 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:54.619210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 404/854] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:35:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:54.619640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56fcd7-fcb2-4560-9957-7593c9b1cba7 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:54.684509 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1ffb6-7ca7-4e6d-b7fb-fce0b8b07692 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:54.727911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56fcd7-fcb2-4560-9957-7593c9b1cba7 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:54.729726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6421c9c-ad24-47fb-8764-02cbaeb681a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:54.730405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6421c9c-ad24-47fb-8764-02cbaeb681a7 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.730690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6421c9c-ad24-47fb-8764-02cbaeb681a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.730977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6421c9c-ad24-47fb-8764-02cbaeb681a7 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.731712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6421c9c-ad24-47fb-8764-02cbaeb681a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:54.732020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6421c9c-ad24-47fb-8764-02cbaeb681a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:54.733410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6421c9c-ad24-47fb-8764-02cbaeb681a7 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:54.751695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 451/855] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:54.753556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6421c9c-ad24-47fb-8764-02cbaeb681a7 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:54.763151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf19d5-1899-4f11-a251-42faa3b9be1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:54.763949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf19d5-1899-4f11-a251-42faa3b9be1f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.764289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf19d5-1899-4f11-a251-42faa3b9be1f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.764654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf19d5-1899-4f11-a251-42faa3b9be1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.772132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf19d5-1899-4f11-a251-42faa3b9be1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:54.773642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf19d5-1899-4f11-a251-42faa3b9be1f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-cbaf19d5-1899-4f11-a251-42faa3b9be1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:54.788250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:54.789873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:54.791178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:54.791178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:54.791178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:54.791178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:54.791178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:54.791178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:54.791178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:54.791178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:54.791178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 405/856] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:54.791178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf19d5-1899-4f11-a251-42faa3b9be1f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:54.815197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:54.869456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:54.881094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6421c9c-ad24-47fb-8764-02cbaeb681a7 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:54.909579 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf19d5-1899-4f11-a251-42faa3b9be1f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:54.917032 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb1ac5-060f-41d5-8b33-0235d174e462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:54.918135 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb1ac5-060f-41d5-8b33-0235d174e462 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.918586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb1ac5-060f-41d5-8b33-0235d174e462 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.919038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb1ac5-060f-41d5-8b33-0235d174e462 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.920265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb1ac5-060f-41d5-8b33-0235d174e462 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:54.921904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb1ac5-060f-41d5-8b33-0235d174e462 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-d7eb1ac5-060f-41d5-8b33-0235d174e462 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:54.936355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:54.937674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:54.938896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 452/857] 10.222.0.70 () {56 vars in 1132 bytes} [Tue Jul 19 20:35:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:54.940066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb1ac5-060f-41d5-8b33-0235d174e462 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:54.953089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8844d4e-f25f-4623-bfe4-5783925a1f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:54.953884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8844d4e-f25f-4623-bfe4-5783925a1f63 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:54.954118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8844d4e-f25f-4623-bfe4-5783925a1f63 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:54.954402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8844d4e-f25f-4623-bfe4-5783925a1f63 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:54.955128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8844d4e-f25f-4623-bfe4-5783925a1f63 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:54.955534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8844d4e-f25f-4623-bfe4-5783925a1f63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:54.956951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8844d4e-f25f-4623-bfe4-5783925a1f63 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:54.962213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 406/858] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:35:54 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:54.963038 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8844d4e-f25f-4623-bfe4-5783925a1f63 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:55.077094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb1ac5-060f-41d5-8b33-0235d174e462 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:55.088169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:55.113990 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8844d4e-f25f-4623-bfe4-5783925a1f63 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:55.122477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c12a0-6986-4500-9359-310b19e252f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:55.123317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c12a0-6986-4500-9359-310b19e252f1 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:55.123604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c12a0-6986-4500-9359-310b19e252f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:55.123890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c12a0-6986-4500-9359-310b19e252f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:55.133066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c12a0-6986-4500-9359-310b19e252f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:55.134586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c12a0-6986-4500-9359-310b19e252f1 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-f93c12a0-6986-4500-9359-310b19e252f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:35:55.151962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:55.153000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 453/859] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jul 19 20:35:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:55.158668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c12a0-6986-4500-9359-310b19e252f1 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:55.162884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432e84d-4f39-4353-bfda-0f4345aef8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:55.163568 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432e84d-4f39-4353-bfda-0f4345aef8cc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:55.163850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432e84d-4f39-4353-bfda-0f4345aef8cc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:55.164124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432e84d-4f39-4353-bfda-0f4345aef8cc None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:55.165006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432e84d-4f39-4353-bfda-0f4345aef8cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:55.165308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432e84d-4f39-4353-bfda-0f4345aef8cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:55.166789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432e84d-4f39-4353-bfda-0f4345aef8cc None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:55.176167 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:55.181108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 407/860] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:55.182129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432e84d-4f39-4353-bfda-0f4345aef8cc None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:55.279091 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c12a0-6986-4500-9359-310b19e252f1 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:55.295657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432e84d-4f39-4353-bfda-0f4345aef8cc None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:55.319025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4c948-8b59-40f7-a1de-33d7a996ff31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:55.319870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4c948-8b59-40f7-a1de-33d7a996ff31 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:55.320531 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4c948-8b59-40f7-a1de-33d7a996ff31 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:55.321327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4c948-8b59-40f7-a1de-33d7a996ff31 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:55.322443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4c948-8b59-40f7-a1de-33d7a996ff31 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:55.322851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4c948-8b59-40f7-a1de-33d7a996ff31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:55.325525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4c948-8b59-40f7-a1de-33d7a996ff31 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:55.332538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212b091-07f7-48db-bcda-44998689ac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:55.333245 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212b091-07f7-48db-bcda-44998689ac0d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:55.333474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212b091-07f7-48db-bcda-44998689ac0d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:55.333954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212b091-07f7-48db-bcda-44998689ac0d None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:55.334815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212b091-07f7-48db-bcda-44998689ac0d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:55.335125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212b091-07f7-48db-bcda-44998689ac0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:55.336775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212b091-07f7-48db-bcda-44998689ac0d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:55.337162 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 454/861] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jul 19 20:35:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:55.338177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4c948-8b59-40f7-a1de-33d7a996ff31 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:55.352484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:55.357683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 408/862] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jul 19 20:35:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:55.359757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212b091-07f7-48db-bcda-44998689ac0d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:55.449950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:55.474324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4c948-8b59-40f7-a1de-33d7a996ff31 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:55.483951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212b091-07f7-48db-bcda-44998689ac0d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:55.516992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f7331-26f4-4919-8092-ceb3463d286b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:55.517801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f7331-26f4-4919-8092-ceb3463d286b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:55.518148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f7331-26f4-4919-8092-ceb3463d286b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:55.518490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f7331-26f4-4919-8092-ceb3463d286b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:55.519403 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f7331-26f4-4919-8092-ceb3463d286b None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=334a380fa68240c79a72746ee4a5f9dc) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:55.519785 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f7331-26f4-4919-8092-ceb3463d286b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:55.521309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f7331-26f4-4919-8092-ceb3463d286b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:55.527926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 455/863] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:55 2022] GET /identity/v3/role_assignments?user.id=334a380fa68240c79a72746ee4a5f9dc&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=85420c68300642aabfd730fb8fef8347 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:55.528746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f7331-26f4-4919-8092-ceb3463d286b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:55.542858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b9d22-97f6-423b-a61f-97771e3f2d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:55.544876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b9d22-97f6-423b-a61f-97771e3f2d94 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:55.545403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b9d22-97f6-423b-a61f-97771e3f2d94 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:55.545873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b9d22-97f6-423b-a61f-97771e3f2d94 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:55.546758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b9d22-97f6-423b-a61f-97771e3f2d94 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:55.547285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b9d22-97f6-423b-a61f-97771e3f2d94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:55.549555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b9d22-97f6-423b-a61f-97771e3f2d94 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:55.561715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 409/864] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jul 19 20:35:55 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:55.676945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f7331-26f4-4919-8092-ceb3463d286b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:55.720289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841328a8-e8ba-4f59-8e25-0a0258843a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:55.721269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841328a8-e8ba-4f59-8e25-0a0258843a67 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:55.721745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841328a8-e8ba-4f59-8e25-0a0258843a67 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:55.722176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841328a8-e8ba-4f59-8e25-0a0258843a67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:55.737965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:55.801625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841328a8-e8ba-4f59-8e25-0a0258843a67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:55.803682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841328a8-e8ba-4f59-8e25-0a0258843a67 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:55.920886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 456/865] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:35:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:35:55.926757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b9d22-97f6-423b-a61f-97771e3f2d94 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:55.998244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:56.038391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b9d22-97f6-423b-a61f-97771e3f2d94 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:56.075399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d62b3-32ee-4675-9df9-6490d98dcc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:56.076312 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d62b3-32ee-4675-9df9-6490d98dcc48 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:56.076587 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d62b3-32ee-4675-9df9-6490d98dcc48 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:56.076913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d62b3-32ee-4675-9df9-6490d98dcc48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:56.089571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d62b3-32ee-4675-9df9-6490d98dcc48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:56.091184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d62b3-32ee-4675-9df9-6490d98dcc48 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-6f5d62b3-32ee-4675-9df9-6490d98dcc48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:35:56.110582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:56.111532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:56.112416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:35:56.112416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:35:56.112416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:35:56.112416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:56.112416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 410/866] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:56.118902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841328a8-e8ba-4f59-8e25-0a0258843a67 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:56.169074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:56.238270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841328a8-e8ba-4f59-8e25-0a0258843a67 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:56.277039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbdc28-5faa-4aed-b8ff-a04e3f5c7f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:56.277652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbdc28-5faa-4aed-b8ff-a04e3f5c7f5d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:56.277937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbdc28-5faa-4aed-b8ff-a04e3f5c7f5d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:56.278212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbdc28-5faa-4aed-b8ff-a04e3f5c7f5d None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:56.278947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbdc28-5faa-4aed-b8ff-a04e3f5c7f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:56.279256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbdc28-5faa-4aed-b8ff-a04e3f5c7f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:56.280695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbdc28-5faa-4aed-b8ff-a04e3f5c7f5d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:56.293700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 457/867] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:56.298921 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d62b3-32ee-4675-9df9-6490d98dcc48 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:56.424144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d62b3-32ee-4675-9df9-6490d98dcc48 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:56.442006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:56.462021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2a319-26e3-4d3a-8b01-ccc61efd16ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:56.462824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2a319-26e3-4d3a-8b01-ccc61efd16ae None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:56.463106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2a319-26e3-4d3a-8b01-ccc61efd16ae None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:56.463386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2a319-26e3-4d3a-8b01-ccc61efd16ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:56.471175 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2a319-26e3-4d3a-8b01-ccc61efd16ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:56.472635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2a319-26e3-4d3a-8b01-ccc61efd16ae None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-f8c2a319-26e3-4d3a-8b01-ccc61efd16ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:35:56.487284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:56.488654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:56.489580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:35:56.489580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:35:56.489580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:35:56.489580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:35:56.489580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:35:56.489580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:35:56.489580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:35:56.489580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:56.489580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 411/868] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:56.496534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbdc28-5faa-4aed-b8ff-a04e3f5c7f5d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:56.600795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbdc28-5faa-4aed-b8ff-a04e3f5c7f5d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:56.602404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:56.644756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad773a-c176-451e-b760-61aafd009d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:56.645474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad773a-c176-451e-b760-61aafd009d8b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:56.645936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad773a-c176-451e-b760-61aafd009d8b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:56.646398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad773a-c176-451e-b760-61aafd009d8b None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:56.647875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad773a-c176-451e-b760-61aafd009d8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:56.648413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad773a-c176-451e-b760-61aafd009d8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:56.650673 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad773a-c176-451e-b760-61aafd009d8b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:56.672428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 458/869] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:56.678650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2a319-26e3-4d3a-8b01-ccc61efd16ae None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:56.792283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2a319-26e3-4d3a-8b01-ccc61efd16ae None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:56.838064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7238de-039c-43f1-bdb0-8fbbff43f8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:56.839407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7238de-039c-43f1-bdb0-8fbbff43f8a8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:56.839767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7238de-039c-43f1-bdb0-8fbbff43f8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:56.840503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7238de-039c-43f1-bdb0-8fbbff43f8a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:56.841626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7238de-039c-43f1-bdb0-8fbbff43f8a8 None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=2711df504d424fa0b05cd16d7fd5acfb) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:56.842027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7238de-039c-43f1-bdb0-8fbbff43f8a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:56.843705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7238de-039c-43f1-bdb0-8fbbff43f8a8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:56.851915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 412/870] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:35:56 2022] GET /identity/v3/role_assignments?user.id=2711df504d424fa0b05cd16d7fd5acfb&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:57.124288 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dabcb-2067-4b57-bb10-cc1b05ea5f2f None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:57.124587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dabcb-2067-4b57-bb10-cc1b05ea5f2f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:57.124774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dabcb-2067-4b57-bb10-cc1b05ea5f2f None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:57.126292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 459/871] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:57.132066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6438-c5b3-4e18-a46e-de747d295b91 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:57.132665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6438-c5b3-4e18-a46e-de747d295b91 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:57.133141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6438-c5b3-4e18-a46e-de747d295b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:57.151735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-cb3b6438-c5b3-4e18-a46e-de747d295b91 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:57.153271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-cb3b6438-c5b3-4e18-a46e-de747d295b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:57.189890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:57.227550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 413/872] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:35:57.250784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dabcb-2067-4b57-bb10-cc1b05ea5f2f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:57.361631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dabcb-2067-4b57-bb10-cc1b05ea5f2f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:57.397509 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac7dbd-3335-466d-9a42-06890686ab23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:57.398105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac7dbd-3335-466d-9a42-06890686ab23 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:57.398390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac7dbd-3335-466d-9a42-06890686ab23 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:57.398664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac7dbd-3335-466d-9a42-06890686ab23 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:57.399405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac7dbd-3335-466d-9a42-06890686ab23 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:57.400775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac7dbd-3335-466d-9a42-06890686ab23 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:57.418585 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 460/873] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:35:57 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:57.831365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cb97d-d6b5-474d-8e96-f9adcdf9a59e None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:57.831927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cb97d-d6b5-474d-8e96-f9adcdf9a59e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:57.832275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cb97d-d6b5-474d-8e96-f9adcdf9a59e None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:57.834314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 414/874] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:57.840314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b045c4-d23f-4fce-ae19-71b929a9f583 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:57.840648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b045c4-d23f-4fce-ae19-71b929a9f583 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:57.840880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b045c4-d23f-4fce-ae19-71b929a9f583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:57.859684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-58b045c4-d23f-4fce-ae19-71b929a9f583 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:57.860908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-58b045c4-d23f-4fce-ae19-71b929a9f583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:57.881824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:57.964895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 461/875] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:57 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:35:57.973027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47f47c-b5b4-4894-a04f-8ed7ef7dbaef None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:57.973343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47f47c-b5b4-4894-a04f-8ed7ef7dbaef None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:57.973619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47f47c-b5b4-4894-a04f-8ed7ef7dbaef None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:57.975421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 415/876] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:57.981752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a17a55-cc8e-4507-97a9-3114513397a8 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:57.982070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a17a55-cc8e-4507-97a9-3114513397a8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:57.982315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a17a55-cc8e-4507-97a9-3114513397a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:57.984161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47f47c-b5b4-4894-a04f-8ed7ef7dbaef None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:58.011321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-c0a17a55-cc8e-4507-97a9-3114513397a8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:58.012554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c0a17a55-cc8e-4507-97a9-3114513397a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:58.151107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 462/877] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:57 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:35:58.157697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47f47c-b5b4-4894-a04f-8ed7ef7dbaef None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:58.177646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a17a55-cc8e-4507-97a9-3114513397a8 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:58.208603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a13849-7aaf-43ce-afc5-482c2bdab3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:58.209310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a13849-7aaf-43ce-afc5-482c2bdab3f6 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:58.209625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a13849-7aaf-43ce-afc5-482c2bdab3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:58.209916 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a13849-7aaf-43ce-afc5-482c2bdab3f6 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:58.211338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a13849-7aaf-43ce-afc5-482c2bdab3f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:58.213054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a13849-7aaf-43ce-afc5-482c2bdab3f6 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:58.241514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 416/878] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:35:57 2022] POST /identity/v3/services => generated 269 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:35:58.303395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:58.337078 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a17a55-cc8e-4507-97a9-3114513397a8 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:58.393486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefebfb-3493-413f-97e9-1df591dd610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:58.394436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefebfb-3493-413f-97e9-1df591dd610d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:58.394730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefebfb-3493-413f-97e9-1df591dd610d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:58.395009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefebfb-3493-413f-97e9-1df591dd610d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:58.395683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eefebfb-3493-413f-97e9-1df591dd610d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:58.397204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eefebfb-3493-413f-97e9-1df591dd610d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-7eefebfb-3493-413f-97e9-1df591dd610d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:58.413407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:58.414617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:35:58.416105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 463/879] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jul 19 20:35:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:35:58.423385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a13849-7aaf-43ce-afc5-482c2bdab3f6 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:58.473733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:58.557444 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a13849-7aaf-43ce-afc5-482c2bdab3f6 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:58.601050 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfc16c-69a3-4e84-9aea-c4f12a5b59b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:58.601770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfc16c-69a3-4e84-9aea-c4f12a5b59b7 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:58.602061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfc16c-69a3-4e84-9aea-c4f12a5b59b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:58.602342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfc16c-69a3-4e84-9aea-c4f12a5b59b7 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:58.603069 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfc16c-69a3-4e84-9aea-c4f12a5b59b7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:58.603382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfc16c-69a3-4e84-9aea-c4f12a5b59b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:58.604880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfc16c-69a3-4e84-9aea-c4f12a5b59b7 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:58.610620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 417/880] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jul 19 20:35:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:58.616494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefebfb-3493-413f-97e9-1df591dd610d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:58.751351 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefebfb-3493-413f-97e9-1df591dd610d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:58.788923 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9aa4b-702a-42d7-af76-840352b23417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:58.789638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9aa4b-702a-42d7-af76-840352b23417 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:58.789923 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9aa4b-702a-42d7-af76-840352b23417 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:58.790298 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9aa4b-702a-42d7-af76-840352b23417 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:58.791027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9aa4b-702a-42d7-af76-840352b23417 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:58.791336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9aa4b-702a-42d7-af76-840352b23417 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:58.792754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9aa4b-702a-42d7-af76-840352b23417 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:58.798577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 464/881] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jul 19 20:35:58 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:59.282883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be10d3-8d6b-4cff-b4a4-7b093ec62e27 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:59.282883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be10d3-8d6b-4cff-b4a4-7b093ec62e27 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:59.282883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be10d3-8d6b-4cff-b4a4-7b093ec62e27 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:59.284176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 418/882] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:35:59.291635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ff67f-bd23-4137-b19c-85500d0fe649 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:59.291635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ff67f-bd23-4137-b19c-85500d0fe649 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:59.291635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ff67f-bd23-4137-b19c-85500d0fe649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:59.296312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:59.318003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-276ff67f-bd23-4137-b19c-85500d0fe649 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:59.319534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-276ff67f-bd23-4137-b19c-85500d0fe649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:59.403215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 465/883] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:35:59.426695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be10d3-8d6b-4cff-b4a4-7b093ec62e27 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:59.538091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:59.544198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be10d3-8d6b-4cff-b4a4-7b093ec62e27 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:59.587949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a257a-009b-406f-a1cd-c4ea2253f1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:59.589428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a257a-009b-406f-a1cd-c4ea2253f1db None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:59.589914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a257a-009b-406f-a1cd-c4ea2253f1db None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:59.590425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a257a-009b-406f-a1cd-c4ea2253f1db None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:59.602344 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a257a-009b-406f-a1cd-c4ea2253f1db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-cd8a257a-009b-406f-a1cd-c4ea2253f1db None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:35:59.610648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:35:59.612385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:35:59.615132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 419/884] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jul 19 20:35:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:35:59.621417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ff67f-bd23-4137-b19c-85500d0fe649 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:59.705383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223bdfb-fb3a-44f0-aed8-bad6ba7def94 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:59.705785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223bdfb-fb3a-44f0-aed8-bad6ba7def94 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:59.706182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223bdfb-fb3a-44f0-aed8-bad6ba7def94 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:59.708620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 420/885] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:35:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:35:59.715345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876352-32e0-4eb8-acd5-bbd51f196944 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:59.715973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876352-32e0-4eb8-acd5-bbd51f196944 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:59.716445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876352-32e0-4eb8-acd5-bbd51f196944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:59.735211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-e9876352-32e0-4eb8-acd5-bbd51f196944 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:35:59.736934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e9876352-32e0-4eb8-acd5-bbd51f196944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:35:59.738060 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ff67f-bd23-4137-b19c-85500d0fe649 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:59.775103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:35:59.784342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0b389-f9bb-480d-bc78-fb6ef30390e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:35:59.785418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0b389-f9bb-480d-bc78-fb6ef30390e7 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:35:59.785927 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0b389-f9bb-480d-bc78-fb6ef30390e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:35:59.786395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0b389-f9bb-480d-bc78-fb6ef30390e7 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:35:59.787345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0b389-f9bb-480d-bc78-fb6ef30390e7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:35:59.787857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0b389-f9bb-480d-bc78-fb6ef30390e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:35:59.790225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0b389-f9bb-480d-bc78-fb6ef30390e7 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:35:59.800612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 466/886] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jul 19 20:35:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:35:59.810982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0b389-f9bb-480d-bc78-fb6ef30390e7 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:59.836992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 421/887] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:35:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:35:59.860883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9876352-32e0-4eb8-acd5-bbd51f196944 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:35:59.973176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0b389-f9bb-480d-bc78-fb6ef30390e7 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:35:59.981643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:00.009566 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9876352-32e0-4eb8-acd5-bbd51f196944 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:00.019262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c068de7-38d8-4073-9f90-a3a18a01c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:00.020701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c068de7-38d8-4073-9f90-a3a18a01c20f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:00.021096 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c068de7-38d8-4073-9f90-a3a18a01c20f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:00.021548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c068de7-38d8-4073-9f90-a3a18a01c20f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:00.038311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c068de7-38d8-4073-9f90-a3a18a01c20f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:00.040626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c068de7-38d8-4073-9f90-a3a18a01c20f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:00.055143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0076dcf-a99c-4000-8f16-b9888f6d0414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:00.056513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0076dcf-a99c-4000-8f16-b9888f6d0414 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:00.057011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0076dcf-a99c-4000-8f16-b9888f6d0414 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:00.058120 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0076dcf-a99c-4000-8f16-b9888f6d0414 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:00.059196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0076dcf-a99c-4000-8f16-b9888f6d0414 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:00.061201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0076dcf-a99c-4000-8f16-b9888f6d0414 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-7c068de7-38d8-4073-9f90-a3a18a01c20f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:00.064513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:00.065836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:00.067369 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:00.067369 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:00.067369 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:00.067369 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:00.068939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 467/888] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:00.077948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c068de7-38d8-4073-9f90-a3a18a01c20f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-e0076dcf-a99c-4000-8f16-b9888f6d0414 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:00.084222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:00.085308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:36:00.086166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:00.088358 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 422/889] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:35:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:00.097699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0076dcf-a99c-4000-8f16-b9888f6d0414 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:00.158681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:00.240088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0076dcf-a99c-4000-8f16-b9888f6d0414 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:00.245576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c068de7-38d8-4073-9f90-a3a18a01c20f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:00.309790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb8298-f22c-4a36-a644-18adc203e86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:00.310789 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb8298-f22c-4a36-a644-18adc203e86e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:00.311287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb8298-f22c-4a36-a644-18adc203e86e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:00.311773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb8298-f22c-4a36-a644-18adc203e86e None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:00.312322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3adec-7a64-4a36-bea7-9cd049080e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:00.313179 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb8298-f22c-4a36-a644-18adc203e86e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:00.313505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3adec-7a64-4a36-bea7-9cd049080e33 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:00.313703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb8298-f22c-4a36-a644-18adc203e86e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:00.313997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3adec-7a64-4a36-bea7-9cd049080e33 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:00.314488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3adec-7a64-4a36-bea7-9cd049080e33 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:00.315823 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3adec-7a64-4a36-bea7-9cd049080e33 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:00.316169 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb8298-f22c-4a36-a644-18adc203e86e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:00.316415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3adec-7a64-4a36-bea7-9cd049080e33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:00.319373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3adec-7a64-4a36-bea7-9cd049080e33 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:00.333560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 423/890] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:36:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:00.341794 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3adec-7a64-4a36-bea7-9cd049080e33 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:00.349525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 468/891] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:00.355127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb8298-f22c-4a36-a644-18adc203e86e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:00.363224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:00.475841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3adec-7a64-4a36-bea7-9cd049080e33 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:00.514930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2170e-d523-458d-8d50-aaa1329006fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:00.515875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2170e-d523-458d-8d50-aaa1329006fa None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:00.516307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2170e-d523-458d-8d50-aaa1329006fa None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:00.516754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2170e-d523-458d-8d50-aaa1329006fa None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:00.518047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2170e-d523-458d-8d50-aaa1329006fa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:00.518525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2170e-d523-458d-8d50-aaa1329006fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:00.520400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2170e-d523-458d-8d50-aaa1329006fa None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:00.527625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 424/892] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:36:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:00.536640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2170e-d523-458d-8d50-aaa1329006fa None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:00.550957 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb8298-f22c-4a36-a644-18adc203e86e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:00.566207 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:00.615215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e984566-ccc7-46f9-afca-31d766c01c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:00.616613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e984566-ccc7-46f9-afca-31d766c01c2e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:00.617116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e984566-ccc7-46f9-afca-31d766c01c2e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:00.617628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e984566-ccc7-46f9-afca-31d766c01c2e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:00.632026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e984566-ccc7-46f9-afca-31d766c01c2e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:00.634881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e984566-ccc7-46f9-afca-31d766c01c2e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:00.662992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-3e984566-ccc7-46f9-afca-31d766c01c2e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:36:00.666472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:00.667470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:00.668398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:00.668398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:00.668398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:36:00.668398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:36:00.668398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:36:00.668398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:36:00.668398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:36:00.668398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:36:00.668398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:36:00.668398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:00.673885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 469/893] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jul 19 20:36:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:00.678731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e984566-ccc7-46f9-afca-31d766c01c2e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:00.706484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2170e-d523-458d-8d50-aaa1329006fa None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:00.767774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408f829-3265-4fe7-81be-eb1d56f05121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:00.768446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408f829-3265-4fe7-81be-eb1d56f05121 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:00.768731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408f829-3265-4fe7-81be-eb1d56f05121 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:00.769003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408f829-3265-4fe7-81be-eb1d56f05121 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:00.769832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408f829-3265-4fe7-81be-eb1d56f05121 None admin] RBAC: Adding query filter params (interface=public, service_id=4cc08bcfabb24432bfd4c27e26ae3d11, region_id=RegionOne) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:00.770144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408f829-3265-4fe7-81be-eb1d56f05121 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:00.771515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408f829-3265-4fe7-81be-eb1d56f05121 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:00.782777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 425/894] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jul 19 20:36:00 2022] GET /identity/v3/endpoints?service_id=4cc08bcfabb24432bfd4c27e26ae3d11&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:00.785500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e2107-856e-4f42-8c33-86c7ee2b07e1 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:00.787359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e2107-856e-4f42-8c33-86c7ee2b07e1 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:00.787359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e2107-856e-4f42-8c33-86c7ee2b07e1 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:00.789883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 426/895] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:00.790666 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e2107-856e-4f42-8c33-86c7ee2b07e1 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:00.823771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e984566-ccc7-46f9-afca-31d766c01c2e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:00.875139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9c720-ca65-431e-bec4-8f11d13fe027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:00.875820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c720-ca65-431e-bec4-8f11d13fe027 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:00.876102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c720-ca65-431e-bec4-8f11d13fe027 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:00.876372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9c720-ca65-431e-bec4-8f11d13fe027 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:00.877220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9c720-ca65-431e-bec4-8f11d13fe027 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:00.877569 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9c720-ca65-431e-bec4-8f11d13fe027 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:00.879085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9c720-ca65-431e-bec4-8f11d13fe027 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:00.902399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 470/896] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jul 19 20:36:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:00.910253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d8d8d-efa5-47ce-888b-e83f046ad4e3 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:00.910621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d8d8d-efa5-47ce-888b-e83f046ad4e3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:00.910954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d8d8d-efa5-47ce-888b-e83f046ad4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:00.928947 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e2107-856e-4f42-8c33-86c7ee2b07e1 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:00.934967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-3d4d8d8d-efa5-47ce-888b-e83f046ad4e3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:00.936163 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3d4d8d8d-efa5-47ce-888b-e83f046ad4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:00.961843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:00.987131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b869b27-7885-4a78-a6a6-3a81bd60a95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:00.988139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b869b27-7885-4a78-a6a6-3a81bd60a95a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:00.988422 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b869b27-7885-4a78-a6a6-3a81bd60a95a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:00.988713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b869b27-7885-4a78-a6a6-3a81bd60a95a None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:00.989449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b869b27-7885-4a78-a6a6-3a81bd60a95a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:00.990859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b869b27-7885-4a78-a6a6-3a81bd60a95a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:00.997739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 427/897] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jul 19 20:36:00 2022] GET /identity/v3/services => generated 1268 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:36:00.998590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b869b27-7885-4a78-a6a6-3a81bd60a95a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:01.015027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:01.061576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 471/898] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:01.102517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d8d8d-efa5-47ce-888b-e83f046ad4e3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:01.188982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b869b27-7885-4a78-a6a6-3a81bd60a95a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:01.242505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe102a7e-9412-4d85-a6f0-0620acb3abc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:01.243179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe102a7e-9412-4d85-a6f0-0620acb3abc4 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:01.243460 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe102a7e-9412-4d85-a6f0-0620acb3abc4 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:01.243740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe102a7e-9412-4d85-a6f0-0620acb3abc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:01.282409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d8d8d-efa5-47ce-888b-e83f046ad4e3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:01.322130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe102a7e-9412-4d85-a6f0-0620acb3abc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:01.323722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe102a7e-9412-4d85-a6f0-0620acb3abc4 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:01.327124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:01.354923 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663d521-7355-4fe7-a6df-812034055618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:01.356203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663d521-7355-4fe7-a6df-812034055618 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:01.356686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663d521-7355-4fe7-a6df-812034055618 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:01.357306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663d521-7355-4fe7-a6df-812034055618 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:01.358707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663d521-7355-4fe7-a6df-812034055618 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:01.361391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663d521-7355-4fe7-a6df-812034055618 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-6663d521-7355-4fe7-a6df-812034055618 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:01.386905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:01.388041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:36:01.388859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:01.393716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 472/899] 10.222.0.70 () {56 vars in 1132 bytes} [Tue Jul 19 20:36:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:01.398550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21d960-8a58-4ef3-bdff-841f645b380d None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:01.398990 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21d960-8a58-4ef3-bdff-841f645b380d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:01.399448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21d960-8a58-4ef3-bdff-841f645b380d None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:01.406386 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 473/900] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:36:01.407816 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21d960-8a58-4ef3-bdff-841f645b380d None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:01.431379 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:01.459186 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 428/901] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:36:00 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:36:01.459841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3debd9-a6fc-4cbc-8bb5-859b9c7f87ec None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:01.460436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3debd9-a6fc-4cbc-8bb5-859b9c7f87ec None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:01.460713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3debd9-a6fc-4cbc-8bb5-859b9c7f87ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:01.478197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-da3debd9-a6fc-4cbc-8bb5-859b9c7f87ec None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:01.480058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-da3debd9-a6fc-4cbc-8bb5-859b9c7f87ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:01.539733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21d960-8a58-4ef3-bdff-841f645b380d None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:01.571936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 429/902] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:01.573171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3debd9-a6fc-4cbc-8bb5-859b9c7f87ec None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:01.591550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862e92d-3100-41cc-be65-d4a34fa4112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:01.593700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862e92d-3100-41cc-be65-d4a34fa4112f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:01.594330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862e92d-3100-41cc-be65-d4a34fa4112f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:01.594938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862e92d-3100-41cc-be65-d4a34fa4112f None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:01.596027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862e92d-3100-41cc-be65-d4a34fa4112f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:01.596347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862e92d-3100-41cc-be65-d4a34fa4112f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:01.597902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862e92d-3100-41cc-be65-d4a34fa4112f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:01.603229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 474/903] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jul 19 20:36:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:01.604010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862e92d-3100-41cc-be65-d4a34fa4112f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:01.633060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:01.723302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3debd9-a6fc-4cbc-8bb5-859b9c7f87ec None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:01.730473 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:01.762961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862e92d-3100-41cc-be65-d4a34fa4112f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:01.769176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ce04b-a25d-4d57-9296-f877ee366a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:01.770416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce04b-a25d-4d57-9296-f877ee366a2d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:01.770869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce04b-a25d-4d57-9296-f877ee366a2d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:01.771296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ce04b-a25d-4d57-9296-f877ee366a2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:01.791651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ce04b-a25d-4d57-9296-f877ee366a2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:01.794846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ce04b-a25d-4d57-9296-f877ee366a2d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:01.827224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a9ee5-b841-46d7-b583-b0e03adc7ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:01.828099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9ee5-b841-46d7-b583-b0e03adc7ba8 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:01.828099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9ee5-b841-46d7-b583-b0e03adc7ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-7f6ce04b-a25d-4d57-9296-f877ee366a2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:01.828518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:01.832365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:01.833239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:01.833239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:01.833239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:01.833239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:01.833508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9ee5-b841-46d7-b583-b0e03adc7ba8 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:01.833508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a9ee5-b841-46d7-b583-b0e03adc7ba8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:01.833635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 430/904] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:01.833635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ce04b-a25d-4d57-9296-f877ee366a2d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:01.835188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a9ee5-b841-46d7-b583-b0e03adc7ba8 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:01.858931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 475/905] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:36:01 2022] POST /identity/v3/services => generated 241 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:36:01.858931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a9ee5-b841-46d7-b583-b0e03adc7ba8 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:01.982993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:02.011865 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ce04b-a25d-4d57-9296-f877ee366a2d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:02.033788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a9ee5-b841-46d7-b583-b0e03adc7ba8 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:02.056116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37a213-79f8-40c7-a452-d62f430c50de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:02.057057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37a213-79f8-40c7-a452-d62f430c50de None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:02.057559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37a213-79f8-40c7-a452-d62f430c50de None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:02.058094 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37a213-79f8-40c7-a452-d62f430c50de None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:02.058970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37a213-79f8-40c7-a452-d62f430c50de None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:02.059451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37a213-79f8-40c7-a452-d62f430c50de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:02.060848 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37a213-79f8-40c7-a452-d62f430c50de None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:02.068710 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 431/906] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jul 19 20:36:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:02.078414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37a213-79f8-40c7-a452-d62f430c50de None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:02.080914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4942a-2afa-4efb-885f-b84630fc404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:02.081654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4942a-2afa-4efb-885f-b84630fc404e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:02.081937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4942a-2afa-4efb-885f-b84630fc404e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:02.082217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4942a-2afa-4efb-885f-b84630fc404e None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:02.083591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4942a-2afa-4efb-885f-b84630fc404e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:02.083591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4942a-2afa-4efb-885f-b84630fc404e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:02.084650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4942a-2afa-4efb-885f-b84630fc404e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:02.093651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:02.100708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 476/907] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:02.111817 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4942a-2afa-4efb-885f-b84630fc404e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:02.232916 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37a213-79f8-40c7-a452-d62f430c50de None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:02.249541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4942a-2afa-4efb-885f-b84630fc404e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:02.275152 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea8d75-b15f-41a1-a9da-d1e2c1a82b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:02.276126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea8d75-b15f-41a1-a9da-d1e2c1a82b9a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:02.277350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea8d75-b15f-41a1-a9da-d1e2c1a82b9a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:02.277669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea8d75-b15f-41a1-a9da-d1e2c1a82b9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:02.278757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea8d75-b15f-41a1-a9da-d1e2c1a82b9a None admin] RBAC: Adding query filter params (interface=public, service_id=20398e20cd154d18a98ed53f886be177, region_id=RegionOne) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:02.279073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea8d75-b15f-41a1-a9da-d1e2c1a82b9a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:02.281435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea8d75-b15f-41a1-a9da-d1e2c1a82b9a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:02.288680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e18ee3-a844-40fc-84e8-bb6d19b064ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:02.289539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e18ee3-a844-40fc-84e8-bb6d19b064ae None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:02.289826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e18ee3-a844-40fc-84e8-bb6d19b064ae None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:02.290110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e18ee3-a844-40fc-84e8-bb6d19b064ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:02.290593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 432/908] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jul 19 20:36:02 2022] GET /identity/v3/endpoints?service_id=20398e20cd154d18a98ed53f886be177&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:02.296729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea8d75-b15f-41a1-a9da-d1e2c1a82b9a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:02.299259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e18ee3-a844-40fc-84e8-bb6d19b064ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:02.300708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e18ee3-a844-40fc-84e8-bb6d19b064ae None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:02.319361 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-55e18ee3-a844-40fc-84e8-bb6d19b064ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:36:02.322261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:02.324561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:02.326099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:36:02.326099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:36:02.326099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:36:02.326099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:36:02.326099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:36:02.326099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:36:02.326099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:02.326099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:02.330107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 477/909] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:36:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:02.336494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e18ee3-a844-40fc-84e8-bb6d19b064ae None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:02.418187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:02.420464 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea8d75-b15f-41a1-a9da-d1e2c1a82b9a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:02.459768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5c4a0-c132-4e43-a996-829b13b4b8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:02.460555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5c4a0-c132-4e43-a996-829b13b4b8d7 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:02.460898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5c4a0-c132-4e43-a996-829b13b4b8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:02.461234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5c4a0-c132-4e43-a996-829b13b4b8d7 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:02.462024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5c4a0-c132-4e43-a996-829b13b4b8d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:02.463502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5c4a0-c132-4e43-a996-829b13b4b8d7 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:02.477522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 433/910] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jul 19 20:36:02 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:36:02.486125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e18ee3-a844-40fc-84e8-bb6d19b064ae None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:02.543266 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20d2f0-cae4-4d51-af1a-847dee9390fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:02.543960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20d2f0-cae4-4d51-af1a-847dee9390fe None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:02.544245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20d2f0-cae4-4d51-af1a-847dee9390fe None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:02.544527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20d2f0-cae4-4d51-af1a-847dee9390fe None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:02.545396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20d2f0-cae4-4d51-af1a-847dee9390fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:02.545732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20d2f0-cae4-4d51-af1a-847dee9390fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:02.547294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20d2f0-cae4-4d51-af1a-847dee9390fe None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:02.564977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 478/911] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:36:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:02.571372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5c4a0-c132-4e43-a996-829b13b4b8d7 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:02.667882 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:02.694072 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5c4a0-c132-4e43-a996-829b13b4b8d7 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:02.734229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229cc83-4a8a-41c4-a48f-ce6c89835d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:02.735110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229cc83-4a8a-41c4-a48f-ce6c89835d17 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:02.735484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229cc83-4a8a-41c4-a48f-ce6c89835d17 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:02.735849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229cc83-4a8a-41c4-a48f-ce6c89835d17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:02.737024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229cc83-4a8a-41c4-a48f-ce6c89835d17 None admin] RBAC: Adding query filter params (role.id=7833471323564c3794e65a6b05374aa4, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=2711df504d424fa0b05cd16d7fd5acfb) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:02.737466 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229cc83-4a8a-41c4-a48f-ce6c89835d17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:02.739470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229cc83-4a8a-41c4-a48f-ce6c89835d17 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:02.748518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 434/912] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:36:02 2022] GET /identity/v3/role_assignments?user.id=2711df504d424fa0b05cd16d7fd5acfb&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=7833471323564c3794e65a6b05374aa4 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:03.554851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e645213-01de-4e62-8ad5-04d92df92ef4 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:03.555670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e645213-01de-4e62-8ad5-04d92df92ef4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:03.555670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e645213-01de-4e62-8ad5-04d92df92ef4 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:03.557196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 479/913] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:36:03.562679 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587024d5-2b40-4631-bb66-4e84fb3ca431 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:03.563342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587024d5-2b40-4631-bb66-4e84fb3ca431 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:03.563342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587024d5-2b40-4631-bb66-4e84fb3ca431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:03.581903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-587024d5-2b40-4631-bb66-4e84fb3ca431 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:03.583131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-587024d5-2b40-4631-bb66-4e84fb3ca431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:03.658382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 435/914] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:03.682026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e645213-01de-4e62-8ad5-04d92df92ef4 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:03.808748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e645213-01de-4e62-8ad5-04d92df92ef4 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:03.836000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:03.849594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2963bc-debb-40ff-a508-189cbe39eb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:03.850398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2963bc-debb-40ff-a508-189cbe39eb68 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:03.850677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2963bc-debb-40ff-a508-189cbe39eb68 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:03.850949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2963bc-debb-40ff-a508-189cbe39eb68 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:03.851599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2963bc-debb-40ff-a508-189cbe39eb68 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:03.852940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2963bc-debb-40ff-a508-189cbe39eb68 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-dd2963bc-debb-40ff-a508-189cbe39eb68 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:03.866306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:03.867345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:36:03.868168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:03.871820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 480/915] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:36:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:03.877662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587024d5-2b40-4631-bb66-4e84fb3ca431 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:03.956707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:03.982696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587024d5-2b40-4631-bb66-4e84fb3ca431 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:04.031454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acd25f-2e29-4a7f-8df6-ec5a3bd669a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:04.032600 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acd25f-2e29-4a7f-8df6-ec5a3bd669a6 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:04.033063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acd25f-2e29-4a7f-8df6-ec5a3bd669a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:04.033535 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acd25f-2e29-4a7f-8df6-ec5a3bd669a6 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:04.034949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acd25f-2e29-4a7f-8df6-ec5a3bd669a6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:04.035495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acd25f-2e29-4a7f-8df6-ec5a3bd669a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:04.037933 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acd25f-2e29-4a7f-8df6-ec5a3bd669a6 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:04.046009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 436/916] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:36:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:04.051642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2963bc-debb-40ff-a508-189cbe39eb68 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:04.187586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2963bc-debb-40ff-a508-189cbe39eb68 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:04.230302 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c36fa-a4fb-4a39-a183-35c66c25ed1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:04.231928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c36fa-a4fb-4a39-a183-35c66c25ed1b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:04.232759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c36fa-a4fb-4a39-a183-35c66c25ed1b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:04.233579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c36fa-a4fb-4a39-a183-35c66c25ed1b None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:04.235039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c36fa-a4fb-4a39-a183-35c66c25ed1b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:04.235892 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c36fa-a4fb-4a39-a183-35c66c25ed1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:04.238706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c36fa-a4fb-4a39-a183-35c66c25ed1b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:04.249257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 481/917] 10.222.0.70 () {56 vars in 1114 bytes} [Tue Jul 19 20:36:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:04.256225 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acd25f-2e29-4a7f-8df6-ec5a3bd669a6 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:04.424737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acd25f-2e29-4a7f-8df6-ec5a3bd669a6 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:04.470447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f0a46-3eab-4ad4-8b13-53b6a9848f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:04.471126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f0a46-3eab-4ad4-8b13-53b6a9848f36 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:04.471407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f0a46-3eab-4ad4-8b13-53b6a9848f36 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:04.471685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f0a46-3eab-4ad4-8b13-53b6a9848f36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:04.472434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f0a46-3eab-4ad4-8b13-53b6a9848f36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:04.473831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f0a46-3eab-4ad4-8b13-53b6a9848f36 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:04.478299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:04.497864 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 437/918] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jul 19 20:36:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:36:04.915433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cebc57-27f6-4c15-bcb0-d1c6001a84c1 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:04.916489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cebc57-27f6-4c15-bcb0-d1c6001a84c1 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:04.917706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cebc57-27f6-4c15-bcb0-d1c6001a84c1 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:04.920512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 482/919] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:04.928656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011612-f9b5-4501-ac97-b0959e7d1d2a None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:04.929705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011612-f9b5-4501-ac97-b0959e7d1d2a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:04.930232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011612-f9b5-4501-ac97-b0959e7d1d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:04.951315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-9a011612-f9b5-4501-ac97-b0959e7d1d2a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:04.952942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-9a011612-f9b5-4501-ac97-b0959e7d1d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:05.020260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db46056-b93f-496b-a3fd-27866af59db2 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:05.021060 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db46056-b93f-496b-a3fd-27866af59db2 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:05.022043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db46056-b93f-496b-a3fd-27866af59db2 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:05.025323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 483/920] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:05.034587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e0268-d478-44fd-82f8-8779da03e9bb None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:05.035498 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e0268-d478-44fd-82f8-8779da03e9bb None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:05.036437 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e0268-d478-44fd-82f8-8779da03e9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:05.046299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 438/921] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:36:05.065600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-744e0268-d478-44fd-82f8-8779da03e9bb None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:05.068018 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-744e0268-d478-44fd-82f8-8779da03e9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:05.071099 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a011612-f9b5-4501-ac97-b0959e7d1d2a None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:05.129167 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:05.181754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 484/922] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:05.205436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e0268-d478-44fd-82f8-8779da03e9bb None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:05.205695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a011612-f9b5-4501-ac97-b0959e7d1d2a None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:05.226435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:05.247879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d86eb0-1819-4164-b73b-814ef1d496d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:05.248689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d86eb0-1819-4164-b73b-814ef1d496d6 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:05.248972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d86eb0-1819-4164-b73b-814ef1d496d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:05.249242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d86eb0-1819-4164-b73b-814ef1d496d6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:05.249931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d86eb0-1819-4164-b73b-814ef1d496d6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:05.251303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d86eb0-1819-4164-b73b-814ef1d496d6 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-a7d86eb0-1819-4164-b73b-814ef1d496d6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:05.266772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:05.269982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:05.270834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 439/923] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jul 19 20:36:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:05.271624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6844ab-a722-45ef-9627-83f64f0d7686 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:05.271624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6844ab-a722-45ef-9627-83f64f0d7686 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:05.271624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6844ab-a722-45ef-9627-83f64f0d7686 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:05.272180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 440/924] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:05.279007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6844ab-a722-45ef-9627-83f64f0d7686 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:05.347347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e0268-d478-44fd-82f8-8779da03e9bb None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:05.396137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192aacab-5197-4dfe-b546-7846cec9ef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:05.397401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192aacab-5197-4dfe-b546-7846cec9ef4f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:05.398346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192aacab-5197-4dfe-b546-7846cec9ef4f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:05.398346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192aacab-5197-4dfe-b546-7846cec9ef4f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:05.399399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192aacab-5197-4dfe-b546-7846cec9ef4f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:05.402225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192aacab-5197-4dfe-b546-7846cec9ef4f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-192aacab-5197-4dfe-b546-7846cec9ef4f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:05.422669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:05.424052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:36:05.425610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:05.429320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 485/925] 10.222.0.70 () {56 vars in 1132 bytes} [Tue Jul 19 20:36:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:05.430550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859fbbe-45a2-41c6-85a5-8c3def2294bf None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:05.430805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859fbbe-45a2-41c6-85a5-8c3def2294bf None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:05.431112 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859fbbe-45a2-41c6-85a5-8c3def2294bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:05.454893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f859fbbe-45a2-41c6-85a5-8c3def2294bf None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:05.456020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f859fbbe-45a2-41c6-85a5-8c3def2294bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:05.457637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6844ab-a722-45ef-9627-83f64f0d7686 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:05.500786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:05.508970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057851a1-4e99-4a8e-ae5f-eb4af4b7d41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:05.509677 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057851a1-4e99-4a8e-ae5f-eb4af4b7d41b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:05.509988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057851a1-4e99-4a8e-ae5f-eb4af4b7d41b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:05.510249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057851a1-4e99-4a8e-ae5f-eb4af4b7d41b None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:05.510995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057851a1-4e99-4a8e-ae5f-eb4af4b7d41b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:05.511751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057851a1-4e99-4a8e-ae5f-eb4af4b7d41b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:05.513148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057851a1-4e99-4a8e-ae5f-eb4af4b7d41b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:05.524112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 441/926] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jul 19 20:36:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:05.524112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057851a1-4e99-4a8e-ae5f-eb4af4b7d41b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:05.558567 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 486/927] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:05.560143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859fbbe-45a2-41c6-85a5-8c3def2294bf None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:05.609634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:05.705141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057851a1-4e99-4a8e-ae5f-eb4af4b7d41b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:05.720624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859fbbe-45a2-41c6-85a5-8c3def2294bf None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:05.769995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f9673-086a-40d5-8486-f8b85432fe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:05.770701 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f9673-086a-40d5-8486-f8b85432fe43 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:05.770991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f9673-086a-40d5-8486-f8b85432fe43 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:05.771309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f9673-086a-40d5-8486-f8b85432fe43 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:05.772034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f9673-086a-40d5-8486-f8b85432fe43 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:05.772307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f9673-086a-40d5-8486-f8b85432fe43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:05.773841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f9673-086a-40d5-8486-f8b85432fe43 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:05.775625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53284bc8-f627-4bd2-b225-1e22a4895923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:05.777202 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53284bc8-f627-4bd2-b225-1e22a4895923 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:05.777667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53284bc8-f627-4bd2-b225-1e22a4895923 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:05.778080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53284bc8-f627-4bd2-b225-1e22a4895923 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:05.779226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53284bc8-f627-4bd2-b225-1e22a4895923 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:05.779697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53284bc8-f627-4bd2-b225-1e22a4895923 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:05.788549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 442/928] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jul 19 20:36:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:05.788824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53284bc8-f627-4bd2-b225-1e22a4895923 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:05.788975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f9673-086a-40d5-8486-f8b85432fe43 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:05.796742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 487/929] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jul 19 20:36:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:05.801554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53284bc8-f627-4bd2-b225-1e22a4895923 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:05.863461 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:05.911375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f9673-086a-40d5-8486-f8b85432fe43 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:05.924983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:05.952504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53284bc8-f627-4bd2-b225-1e22a4895923 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:05.960805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cff447-c563-411d-838c-610a05e3bca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:05.962143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cff447-c563-411d-838c-610a05e3bca8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:05.962659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cff447-c563-411d-838c-610a05e3bca8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:05.963156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cff447-c563-411d-838c-610a05e3bca8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:05.980278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cff447-c563-411d-838c-610a05e3bca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-42cff447-c563-411d-838c-610a05e3bca8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:36:05.990162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:05.991135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 443/930] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jul 19 20:36:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:05.997935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cff447-c563-411d-838c-610a05e3bca8 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:06.009511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b01c9-82a9-426a-8dd7-6587ce15a613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:06.010187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b01c9-82a9-426a-8dd7-6587ce15a613 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:06.010503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b01c9-82a9-426a-8dd7-6587ce15a613 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:06.010776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b01c9-82a9-426a-8dd7-6587ce15a613 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:06.011487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b01c9-82a9-426a-8dd7-6587ce15a613 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:06.011790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b01c9-82a9-426a-8dd7-6587ce15a613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:06.013146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b01c9-82a9-426a-8dd7-6587ce15a613 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:06.023698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 488/931] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jul 19 20:36:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:06.023698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b01c9-82a9-426a-8dd7-6587ce15a613 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:06.162130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cff447-c563-411d-838c-610a05e3bca8 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:06.179632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:06.187333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b01c9-82a9-426a-8dd7-6587ce15a613 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:06.215143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f41f3e-d62d-43c1-83b9-25d23b16f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:06.216542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f41f3e-d62d-43c1-83b9-25d23b16f765 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:06.216987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f41f3e-d62d-43c1-83b9-25d23b16f765 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:06.217474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f41f3e-d62d-43c1-83b9-25d23b16f765 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:06.218875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f41f3e-d62d-43c1-83b9-25d23b16f765 None admin] RBAC: Adding query filter params (interface=public, service_id=fe768e6693574f05a864de3b9789557c, region_id=RegionOne) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:06.219365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f41f3e-d62d-43c1-83b9-25d23b16f765 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:06.221622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f41f3e-d62d-43c1-83b9-25d23b16f765 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:06.232612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 444/932] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jul 19 20:36:05 2022] GET /identity/v3/endpoints?service_id=fe768e6693574f05a864de3b9789557c&interface=public®ion_id=RegionOne => generated 194 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:06.233401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f41f3e-d62d-43c1-83b9-25d23b16f765 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:06.241560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c931d1-5a16-46b7-a765-f7501c509161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:06.242243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c931d1-5a16-46b7-a765-f7501c509161 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:06.242529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c931d1-5a16-46b7-a765-f7501c509161 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:06.242814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c931d1-5a16-46b7-a765-f7501c509161 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:06.243573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c931d1-5a16-46b7-a765-f7501c509161 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:06.243887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c931d1-5a16-46b7-a765-f7501c509161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:06.245846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c931d1-5a16-46b7-a765-f7501c509161 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:06.255373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 489/933] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jul 19 20:36:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:06.261244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c931d1-5a16-46b7-a765-f7501c509161 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:06.288037 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:06.375977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f41f3e-d62d-43c1-83b9-25d23b16f765 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:06.426850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c5f32-2ce0-44d8-b69b-6a28d18b36fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:06.427531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c5f32-2ce0-44d8-b69b-6a28d18b36fa None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:06.427813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c5f32-2ce0-44d8-b69b-6a28d18b36fa None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:06.428087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c5f32-2ce0-44d8-b69b-6a28d18b36fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:06.428907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c5f32-2ce0-44d8-b69b-6a28d18b36fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:06.430340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c5f32-2ce0-44d8-b69b-6a28d18b36fa None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:06.440954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c931d1-5a16-46b7-a765-f7501c509161 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:06.454059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 445/934] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jul 19 20:36:06 2022] POST /identity/v3/endpoints => generated 347 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:36:06.454900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c5f32-2ce0-44d8-b69b-6a28d18b36fa None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:06.520702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200cfbe-37ee-4541-9eed-1d4704c1ab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:06.521554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200cfbe-37ee-4541-9eed-1d4704c1ab5b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:06.521947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200cfbe-37ee-4541-9eed-1d4704c1ab5b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:06.522325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200cfbe-37ee-4541-9eed-1d4704c1ab5b None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:06.523184 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200cfbe-37ee-4541-9eed-1d4704c1ab5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:06.524757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200cfbe-37ee-4541-9eed-1d4704c1ab5b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:06.532738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 490/935] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jul 19 20:36:06 2022] GET /identity/v3/services => generated 1497 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:36:06.625220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:06.644429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c5f32-2ce0-44d8-b69b-6a28d18b36fa None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:06.692359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f0fd7-2329-4657-8d6f-5455630ead28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:06.693642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f0fd7-2329-4657-8d6f-5455630ead28 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:06.694091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f0fd7-2329-4657-8d6f-5455630ead28 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:06.694814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f0fd7-2329-4657-8d6f-5455630ead28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:06.708597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f0fd7-2329-4657-8d6f-5455630ead28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:06.710430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f0fd7-2329-4657-8d6f-5455630ead28 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-f74f0fd7-2329-4657-8d6f-5455630ead28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:06.736196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:06.737223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:06.738208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:06.738208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:06.738208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:06.738208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:06.738208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 446/936] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:06.743345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200cfbe-37ee-4541-9eed-1d4704c1ab5b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:06.770082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:06.771078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed9083-373f-4f95-b800-3f002abc857c None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:06.771460 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed9083-373f-4f95-b800-3f002abc857c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:06.771804 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed9083-373f-4f95-b800-3f002abc857c None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:06.774055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 447/937] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:06.779732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccfca-4124-442b-9e31-f9c3a7c300ac None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:06.780031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccfca-4124-442b-9e31-f9c3a7c300ac None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:06.780313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccfca-4124-442b-9e31-f9c3a7c300ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:06.799708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-307ccfca-4124-442b-9e31-f9c3a7c300ac None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:06.800943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-307ccfca-4124-442b-9e31-f9c3a7c300ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:06.869828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200cfbe-37ee-4541-9eed-1d4704c1ab5b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:06.905443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 448/938] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:36:06.935486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ccfca-4124-442b-9e31-f9c3a7c300ac None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:06.939820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde7829-24f1-4588-a808-ac08e4ca7027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:06.940700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde7829-24f1-4588-a808-ac08e4ca7027 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:06.941148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde7829-24f1-4588-a808-ac08e4ca7027 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:06.941617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde7829-24f1-4588-a808-ac08e4ca7027 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:06.942540 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde7829-24f1-4588-a808-ac08e4ca7027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:06.943022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde7829-24f1-4588-a808-ac08e4ca7027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:06.944538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde7829-24f1-4588-a808-ac08e4ca7027 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:06.971207 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 491/939] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:06.981005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde7829-24f1-4588-a808-ac08e4ca7027 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:07.074928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:07.081914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:07.111081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde7829-24f1-4588-a808-ac08e4ca7027 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:07.117575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ccfca-4124-442b-9e31-f9c3a7c300ac None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:07.150753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101de81a-3350-41a2-bb7c-8a0f15bf1d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:07.152603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101de81a-3350-41a2-bb7c-8a0f15bf1d02 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:07.153012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101de81a-3350-41a2-bb7c-8a0f15bf1d02 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:07.153448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101de81a-3350-41a2-bb7c-8a0f15bf1d02 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:07.163864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101de81a-3350-41a2-bb7c-8a0f15bf1d02 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:07.165911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101de81a-3350-41a2-bb7c-8a0f15bf1d02 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:07.184465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc88446-b249-4c45-ab85-8cfad9642a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:07.185564 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc88446-b249-4c45-ab85-8cfad9642a58 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-101de81a-3350-41a2-bb7c-8a0f15bf1d02 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:36:07.185744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:07.186815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:07.187872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:07.187872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:07.187872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:36:07.187872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:36:07.187872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:36:07.187872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:36:07.187872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:36:07.187872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:36:07.187872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:36:07.187872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:07.187872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 492/940] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jul 19 20:36:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:07.189622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc88446-b249-4c45-ab85-8cfad9642a58 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:07.190223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc88446-b249-4c45-ab85-8cfad9642a58 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:07.191149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc88446-b249-4c45-ab85-8cfad9642a58 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:07.192765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc88446-b249-4c45-ab85-8cfad9642a58 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:07.194383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: WARNING keystone.server.flask.common [None req-9bc88446-b249-4c45-ab85-8cfad9642a58 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 19 20:36:07.195595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101de81a-3350-41a2-bb7c-8a0f15bf1d02 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:07.219924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.sql.core [None req-9bc88446-b249-4c45-ab85-8cfad9642a58 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 20:36:07.219924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 19 20:36:07.219924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [parameters: {'id': 'fac382a599b548de9a49bfa106d99d3d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 19 20:36:07.219924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92311) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-9bc88446-b249-4c45-ab85-8cfad9642a58 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fac382a599b548de9a49bfa106d99d3d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fac382a599b548de9a49bfa106d99d3d. Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:36:07.244169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application result.read() Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 19 20:36:07.245089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application yield resource Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise self.value Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application session.commit() Jul 19 20:36:07.246098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self.session.flush() Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application compat.raise_( Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 20:36:07.246982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 20:36:07.247871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application result.read() Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:36:07.250061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [parameters: {'id': 'fac382a599b548de9a49bfa106d99d3d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:07.251200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:07.253311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:07.255638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:07.255638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 19 20:36:07.255638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 19 20:36:07.255638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:07.255638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:07.255638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 19 20:36:07.255638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 19 20:36:07.255638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fac382a599b548de9a49bfa106d99d3d. Jul 19 20:36:07.255638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:07.255638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 449/941] 10.222.0.70 () {60 vars in 1140 bytes} [Tue Jul 19 20:36:06 2022] POST /identity/v3/projects => generated 262 bytes in 313 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:07.255638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc88446-b249-4c45-ab85-8cfad9642a58 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:07.337862 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101de81a-3350-41a2-bb7c-8a0f15bf1d02 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:07.406388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7aedcd-b0fe-4358-977c-7e1b319cb39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:07.407479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7aedcd-b0fe-4358-977c-7e1b319cb39e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:07.407872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7aedcd-b0fe-4358-977c-7e1b319cb39e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:07.408276 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7aedcd-b0fe-4358-977c-7e1b319cb39e None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:07.409491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7aedcd-b0fe-4358-977c-7e1b319cb39e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:07.409909 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7aedcd-b0fe-4358-977c-7e1b319cb39e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:07.411055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc88446-b249-4c45-ab85-8cfad9642a58 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:07.412273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7aedcd-b0fe-4358-977c-7e1b319cb39e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:07.429492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:07.446885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 493/942] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jul 19 20:36:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:07.448767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7aedcd-b0fe-4358-977c-7e1b319cb39e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:07.461081 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:07.475730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a99db7-c08b-48f7-b4ac-a245f353c4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:07.477406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99db7-c08b-48f7-b4ac-a245f353c4ea None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:07.477880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99db7-c08b-48f7-b4ac-a245f353c4ea None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:07.478351 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99db7-c08b-48f7-b4ac-a245f353c4ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:07.492706 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a99db7-c08b-48f7-b4ac-a245f353c4ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:07.496881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a99db7-c08b-48f7-b4ac-a245f353c4ea None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-a7a99db7-c08b-48f7-b4ac-a245f353c4ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:36:07.527509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:07.529247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:07.530703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:36:07.530703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:36:07.530703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:36:07.530703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:36:07.530703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:36:07.530703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:36:07.530703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:07.530703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:07.530703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 450/943] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:36:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:07.537154 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a99db7-c08b-48f7-b4ac-a245f353c4ea None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:07.615407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7aedcd-b0fe-4358-977c-7e1b319cb39e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:07.670823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bd782-a33b-44d4-904e-c0cb57c0c21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:07.671880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd782-a33b-44d4-904e-c0cb57c0c21a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:07.672420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd782-a33b-44d4-904e-c0cb57c0c21a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:07.673156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd782-a33b-44d4-904e-c0cb57c0c21a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:07.676248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a99db7-c08b-48f7-b4ac-a245f353c4ea None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:07.686294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bd782-a33b-44d4-904e-c0cb57c0c21a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:07.688203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bd782-a33b-44d4-904e-c0cb57c0c21a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-cc6bd782-a33b-44d4-904e-c0cb57c0c21a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:07.710121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:07.711182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:07.712112 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:07.712112 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:07.712112 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:07.712112 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:07.720153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 494/944] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:07.721632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bd782-a33b-44d4-904e-c0cb57c0c21a None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:07.723264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e738a-6a53-4f71-b603-bd2a9d2eb0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:07.724004 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e738a-6a53-4f71-b603-bd2a9d2eb0f4 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:07.724519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e738a-6a53-4f71-b603-bd2a9d2eb0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:07.724805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e738a-6a53-4f71-b603-bd2a9d2eb0f4 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:07.725862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e738a-6a53-4f71-b603-bd2a9d2eb0f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:07.726217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e738a-6a53-4f71-b603-bd2a9d2eb0f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:07.728010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e738a-6a53-4f71-b603-bd2a9d2eb0f4 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:07.754091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 451/945] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:36:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:07.811299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:07.887807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bd782-a33b-44d4-904e-c0cb57c0c21a None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:07.932063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c3de6-055c-4976-b1b4-a6c3897cbad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:07.932801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c3de6-055c-4976-b1b4-a6c3897cbad2 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:07.933088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c3de6-055c-4976-b1b4-a6c3897cbad2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:07.933392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c3de6-055c-4976-b1b4-a6c3897cbad2 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:07.934153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c3de6-055c-4976-b1b4-a6c3897cbad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:07.934463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c3de6-055c-4976-b1b4-a6c3897cbad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:07.935866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c3de6-055c-4976-b1b4-a6c3897cbad2 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:07.951334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 495/946] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:07.957950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e738a-6a53-4f71-b603-bd2a9d2eb0f4 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:08.029462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:08.098419 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e738a-6a53-4f71-b603-bd2a9d2eb0f4 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:08.140529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62405adc-c4f2-414d-9a1f-842baa727cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:08.141306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62405adc-c4f2-414d-9a1f-842baa727cac None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:08.141734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62405adc-c4f2-414d-9a1f-842baa727cac None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:08.142156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62405adc-c4f2-414d-9a1f-842baa727cac None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:08.150374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62405adc-c4f2-414d-9a1f-842baa727cac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:08.152775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62405adc-c4f2-414d-9a1f-842baa727cac None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-62405adc-c4f2-414d-9a1f-842baa727cac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:36:08.171357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:08.172310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:08.173402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:36:08.173402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:36:08.173402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:36:08.173402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:36:08.173402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:36:08.173402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:36:08.173402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:08.173402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:08.173402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 452/947] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:36:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:08.181028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c3de6-055c-4976-b1b4-a6c3897cbad2 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:08.311753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c3de6-055c-4976-b1b4-a6c3897cbad2 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:08.327055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:08.363106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e05077-56c9-4bd9-9fec-c438cb782b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:08.363974 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e05077-56c9-4bd9-9fec-c438cb782b9a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:08.364491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e05077-56c9-4bd9-9fec-c438cb782b9a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:08.364959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e05077-56c9-4bd9-9fec-c438cb782b9a None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:08.366528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e05077-56c9-4bd9-9fec-c438cb782b9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:08.366891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e05077-56c9-4bd9-9fec-c438cb782b9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:08.371659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e05077-56c9-4bd9-9fec-c438cb782b9a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:08.391883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 496/948] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:36:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:08.397820 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62405adc-c4f2-414d-9a1f-842baa727cac None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:08.597649 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62405adc-c4f2-414d-9a1f-842baa727cac None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:08.625786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:08.662539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d7719-5d8b-426b-9763-3d6f280438e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:08.664585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d7719-5d8b-426b-9763-3d6f280438e0 None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:08.665383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d7719-5d8b-426b-9763-3d6f280438e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:08.666156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d7719-5d8b-426b-9763-3d6f280438e0 None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/2711df504d424fa0b05cd16d7fd5acfb/roles/7833471323564c3794e65a6b05374aa4` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:08.671057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d7719-5d8b-426b-9763-3d6f280438e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=2711df504d424fa0b05cd16d7fd5acfb, role_id=7833471323564c3794e65a6b05374aa4)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:08.673996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d7719-5d8b-426b-9763-3d6f280438e0 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:08.700354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 453/949] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:36:08 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/2711df504d424fa0b05cd16d7fd5acfb/roles/7833471323564c3794e65a6b05374aa4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:36:09.320751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c35e7-c0f8-442f-99c8-909a53acd552 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:09.321600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c35e7-c0f8-442f-99c8-909a53acd552 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:09.321738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c35e7-c0f8-442f-99c8-909a53acd552 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:09.325015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 497/950] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:09.334275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ad0e2-657d-47ad-add8-db842c816eb2 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:09.335088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ad0e2-657d-47ad-add8-db842c816eb2 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:09.335238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ad0e2-657d-47ad-add8-db842c816eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:09.364009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-180ad0e2-657d-47ad-add8-db842c816eb2 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:09.366557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-180ad0e2-657d-47ad-add8-db842c816eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:09.384923 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0785f-96b8-4d44-9f7b-64db8fdc13f6 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:09.385201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0785f-96b8-4d44-9f7b-64db8fdc13f6 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:09.385569 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0785f-96b8-4d44-9f7b-64db8fdc13f6 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:09.387087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 498/951] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:09.393709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c186c8f-5e6e-46c4-8c39-8f9d307151d9 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:09.393992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c186c8f-5e6e-46c4-8c39-8f9d307151d9 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:09.394416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c186c8f-5e6e-46c4-8c39-8f9d307151d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:09.412649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-6c186c8f-5e6e-46c4-8c39-8f9d307151d9 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:09.413966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6c186c8f-5e6e-46c4-8c39-8f9d307151d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:09.458122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 454/952] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:09.481244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ad0e2-657d-47ad-add8-db842c816eb2 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:09.490505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 499/953] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:09.513601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c186c8f-5e6e-46c4-8c39-8f9d307151d9 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:09.607089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ad0e2-657d-47ad-add8-db842c816eb2 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:09.629801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:09.638223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:09.643430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c186c8f-5e6e-46c4-8c39-8f9d307151d9 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:09.651889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258bcc-7b31-4df4-a278-8e4d53e3dace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:09.653515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258bcc-7b31-4df4-a278-8e4d53e3dace None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:09.654176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258bcc-7b31-4df4-a278-8e4d53e3dace None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:09.654929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258bcc-7b31-4df4-a278-8e4d53e3dace None admin] PATH_INFO: `/v3/services/compute` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:09.655819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5258bcc-7b31-4df4-a278-8e4d53e3dace None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:09.657629 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5258bcc-7b31-4df4-a278-8e4d53e3dace None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-b5258bcc-7b31-4df4-a278-8e4d53e3dace None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:09.672215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:09.673408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:36:09.674486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:09.677401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 455/954] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:36:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:09.684415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258bcc-7b31-4df4-a278-8e4d53e3dace None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:09.689318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac5b12-8b1d-42b9-8857-d8befdba0820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:09.690576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac5b12-8b1d-42b9-8857-d8befdba0820 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:09.691115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac5b12-8b1d-42b9-8857-d8befdba0820 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:09.691597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac5b12-8b1d-42b9-8857-d8befdba0820 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:09.692549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac5b12-8b1d-42b9-8857-d8befdba0820 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:09.694233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac5b12-8b1d-42b9-8857-d8befdba0820 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-ebac5b12-8b1d-42b9-8857-d8befdba0820 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:09.708574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:09.710016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:09.711432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:09.721395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 500/955] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jul 19 20:36:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:09.725645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac5b12-8b1d-42b9-8857-d8befdba0820 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:09.796803 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258bcc-7b31-4df4-a278-8e4d53e3dace None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:09.836836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abafd41-3b3c-45e9-b0ac-a1e1112fab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:09.837535 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abafd41-3b3c-45e9-b0ac-a1e1112fab44 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:09.837825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abafd41-3b3c-45e9-b0ac-a1e1112fab44 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:09.838104 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abafd41-3b3c-45e9-b0ac-a1e1112fab44 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:09.839399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abafd41-3b3c-45e9-b0ac-a1e1112fab44 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:09.839713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abafd41-3b3c-45e9-b0ac-a1e1112fab44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:09.841201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abafd41-3b3c-45e9-b0ac-a1e1112fab44 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:09.847094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac5b12-8b1d-42b9-8857-d8befdba0820 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:09.851175 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 456/956] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:36:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:09.852568 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abafd41-3b3c-45e9-b0ac-a1e1112fab44 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:09.883508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:09.888786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ef864-70f3-49d7-ad97-5163a2886a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:09.889918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ef864-70f3-49d7-ad97-5163a2886a66 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:09.890657 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ef864-70f3-49d7-ad97-5163a2886a66 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:09.891446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ef864-70f3-49d7-ad97-5163a2886a66 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:09.893047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ef864-70f3-49d7-ad97-5163a2886a66 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:09.894016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ef864-70f3-49d7-ad97-5163a2886a66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:09.897076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ef864-70f3-49d7-ad97-5163a2886a66 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:09.908747 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 501/957] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jul 19 20:36:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:09.910047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ef864-70f3-49d7-ad97-5163a2886a66 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:09.967575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abafd41-3b3c-45e9-b0ac-a1e1112fab44 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:09.973774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:10.005139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54323e89-d747-46e4-bb1f-c13b4cfa486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:10.005856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54323e89-d747-46e4-bb1f-c13b4cfa486f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:10.006165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54323e89-d747-46e4-bb1f-c13b4cfa486f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:10.006423 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54323e89-d747-46e4-bb1f-c13b4cfa486f None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:10.007144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54323e89-d747-46e4-bb1f-c13b4cfa486f None admin] RBAC: Adding query filter params (type=compute) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:10.007713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54323e89-d747-46e4-bb1f-c13b4cfa486f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:10.008849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54323e89-d747-46e4-bb1f-c13b4cfa486f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:10.021517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 457/958] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:36:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:10.030174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ef864-70f3-49d7-ad97-5163a2886a66 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:10.072519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f17a7-691a-4f12-a7a4-0920e80331f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:10.074115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f17a7-691a-4f12-a7a4-0920e80331f0 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:10.074914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f17a7-691a-4f12-a7a4-0920e80331f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:10.075348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f17a7-691a-4f12-a7a4-0920e80331f0 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:10.076254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f17a7-691a-4f12-a7a4-0920e80331f0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:10.076721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f17a7-691a-4f12-a7a4-0920e80331f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:10.078326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f17a7-691a-4f12-a7a4-0920e80331f0 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:10.084945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 502/959] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jul 19 20:36:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:10.091342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54323e89-d747-46e4-bb1f-c13b4cfa486f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:10.134321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831565f-0fd9-474b-a2ea-22fde8b2f8ca None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:10.134940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831565f-0fd9-474b-a2ea-22fde8b2f8ca None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:10.136068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831565f-0fd9-474b-a2ea-22fde8b2f8ca None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:10.138029 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 503/960] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:36:10.146925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad724f4e-1c8e-4969-9271-767332e4b770 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:10.147671 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad724f4e-1c8e-4969-9271-767332e4b770 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:10.148178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad724f4e-1c8e-4969-9271-767332e4b770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:10.167678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-ad724f4e-1c8e-4969-9271-767332e4b770 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:10.169429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ad724f4e-1c8e-4969-9271-767332e4b770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:10.203990 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:10.219002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54323e89-d747-46e4-bb1f-c13b4cfa486f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:10.267642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158502d-1211-4804-8267-3b45e5a82d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:10.268420 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158502d-1211-4804-8267-3b45e5a82d9c None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:10.268763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158502d-1211-4804-8267-3b45e5a82d9c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:10.269108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158502d-1211-4804-8267-3b45e5a82d9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:10.269978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158502d-1211-4804-8267-3b45e5a82d9c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:10.271514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158502d-1211-4804-8267-3b45e5a82d9c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:10.274741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 504/961] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:10.296059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 458/962] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jul 19 20:36:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:36:10.301462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad724f4e-1c8e-4969-9271-767332e4b770 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:10.331898 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:10.430504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad724f4e-1c8e-4969-9271-767332e4b770 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:10.477777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0e412-babe-4092-96cd-63c0d187b3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:10.478472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0e412-babe-4092-96cd-63c0d187b3ec None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:10.478753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0e412-babe-4092-96cd-63c0d187b3ec None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:10.479722 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0e412-babe-4092-96cd-63c0d187b3ec None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:10.480463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0e412-babe-4092-96cd-63c0d187b3ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:10.481954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0e412-babe-4092-96cd-63c0d187b3ec None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:10.483293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: WARNING keystone.server.flask.common [None req-e5f0e412-babe-4092-96cd-63c0d187b3ec None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 19 20:36:10.498264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.sql.core [None req-e5f0e412-babe-4092-96cd-63c0d187b3ec None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 20:36:10.498264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 19 20:36:10.498264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [parameters: {'user_id': '2bfffa6aed424789bd8ce6d484b64e49', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 19 20:36:10.498264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92310) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-e5f0e412-babe-4092-96cd-63c0d187b3ec None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application result.read() Jul 19 20:36:10.506210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:36:10.507168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application yield resource Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise self.value Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application session.commit() Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 20:36:10.508435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self.session.flush() Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application compat.raise_( Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 20:36:10.510704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:36:10.513280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application result.read() Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:36:10.515840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [parameters: {'user_id': '2bfffa6aed424789bd8ce6d484b64e49', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:10.517939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 19 20:36:10.519845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 19 20:36:10.521982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 19 20:36:10.521982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 19 20:36:10.521982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 20:36:10.521982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 20:36:10.521982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 19 20:36:10.521982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:10.521982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 505/963] 10.222.0.70 () {60 vars in 1131 bytes} [Tue Jul 19 20:36:10 2022] POST /identity/v3/users => generated 160 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:10.523069 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158502d-1211-4804-8267-3b45e5a82d9c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:10.659924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158502d-1211-4804-8267-3b45e5a82d9c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:10.712517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6626005-d516-44df-92c1-d70fdb8e257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:10.714611 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6626005-d516-44df-92c1-d70fdb8e257f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:10.715294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6626005-d516-44df-92c1-d70fdb8e257f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:10.715900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6626005-d516-44df-92c1-d70fdb8e257f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:10.728797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6626005-d516-44df-92c1-d70fdb8e257f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:10.731175 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6626005-d516-44df-92c1-d70fdb8e257f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-f6626005-d516-44df-92c1-d70fdb8e257f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:36:10.747222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:10.748229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:10.749141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:10.749141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:10.749141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:36:10.749141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:36:10.749141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:36:10.749141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:36:10.749141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:36:10.749141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:36:10.749141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:36:10.749141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:10.752282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 459/964] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:36:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:10.758038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0e412-babe-4092-96cd-63c0d187b3ec None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:10.869195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:10.891310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0e412-babe-4092-96cd-63c0d187b3ec None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:10.928335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380b84d-77a6-49d0-a70f-787e36e05643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:10.929016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380b84d-77a6-49d0-a70f-787e36e05643 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:10.929303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380b84d-77a6-49d0-a70f-787e36e05643 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:10.929618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380b84d-77a6-49d0-a70f-787e36e05643 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:10.930439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380b84d-77a6-49d0-a70f-787e36e05643 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:10.930747 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380b84d-77a6-49d0-a70f-787e36e05643 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:10.932427 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380b84d-77a6-49d0-a70f-787e36e05643 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:10.975584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 506/965] 10.222.0.70 () {56 vars in 1101 bytes} [Tue Jul 19 20:36:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:11.014236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883b889-2a50-48b4-9d0a-a6c0c4a360fc None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:11.014546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883b889-2a50-48b4-9d0a-a6c0c4a360fc None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:11.014791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883b889-2a50-48b4-9d0a-a6c0c4a360fc None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:11.016567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 460/966] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:11.022324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6d4d6-a6aa-458e-97d1-c627f26ba28a None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:11.022617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6d4d6-a6aa-458e-97d1-c627f26ba28a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:11.022897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6d4d6-a6aa-458e-97d1-c627f26ba28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:11.042365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-56f6d4d6-a6aa-458e-97d1-c627f26ba28a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:11.043983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-56f6d4d6-a6aa-458e-97d1-c627f26ba28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:11.120473 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 507/967] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:11.144812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883b889-2a50-48b4-9d0a-a6c0c4a360fc None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:11.157048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:11.262774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883b889-2a50-48b4-9d0a-a6c0c4a360fc None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:11.301889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79dd9f-31e4-4cae-93b7-c3774d5b9c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:11.302711 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79dd9f-31e4-4cae-93b7-c3774d5b9c52 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:11.302997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79dd9f-31e4-4cae-93b7-c3774d5b9c52 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:11.303276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79dd9f-31e4-4cae-93b7-c3774d5b9c52 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:11.314706 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79dd9f-31e4-4cae-93b7-c3774d5b9c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-ce79dd9f-31e4-4cae-93b7-c3774d5b9c52 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:36:11.323640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:11.324819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:11.326092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 461/968] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jul 19 20:36:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:11.332812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6d4d6-a6aa-458e-97d1-c627f26ba28a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:11.405435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:11.440636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6d4d6-a6aa-458e-97d1-c627f26ba28a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:11.482275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fe461-abdf-4732-9593-9fde93e236a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:11.482950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fe461-abdf-4732-9593-9fde93e236a7 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:11.483236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fe461-abdf-4732-9593-9fde93e236a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:11.483519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fe461-abdf-4732-9593-9fde93e236a7 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:11.484271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fe461-abdf-4732-9593-9fde93e236a7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:11.484581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fe461-abdf-4732-9593-9fde93e236a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:11.485977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fe461-abdf-4732-9593-9fde93e236a7 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:11.500172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 508/969] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jul 19 20:36:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:11.501502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79dd9f-31e4-4cae-93b7-c3774d5b9c52 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:11.568895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:11.605306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79dd9f-31e4-4cae-93b7-c3774d5b9c52 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:11.640642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631863e-486f-432d-8c81-b00890348e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:11.641532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631863e-486f-432d-8c81-b00890348e47 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:11.641815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631863e-486f-432d-8c81-b00890348e47 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:11.642133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631863e-486f-432d-8c81-b00890348e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:11.652479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631863e-486f-432d-8c81-b00890348e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:11.653951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631863e-486f-432d-8c81-b00890348e47 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-0631863e-486f-432d-8c81-b00890348e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:11.671434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:11.672347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:11.678278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:11.678278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:11.678278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:11.678278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:11.678278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 462/970] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:11.678565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fe461-abdf-4732-9593-9fde93e236a7 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:11.779577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fe461-abdf-4732-9593-9fde93e236a7 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:11.808242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:11.815211 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb588b8-2ed5-4d18-9b08-ff15fd0c38e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:11.815931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb588b8-2ed5-4d18-9b08-ff15fd0c38e4 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:11.816217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb588b8-2ed5-4d18-9b08-ff15fd0c38e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:11.816490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb588b8-2ed5-4d18-9b08-ff15fd0c38e4 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:11.817209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb588b8-2ed5-4d18-9b08-ff15fd0c38e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:11.817544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb588b8-2ed5-4d18-9b08-ff15fd0c38e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:11.818894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb588b8-2ed5-4d18-9b08-ff15fd0c38e4 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:11.836842 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 509/971] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:11.837980 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631863e-486f-432d-8c81-b00890348e47 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:11.944787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631863e-486f-432d-8c81-b00890348e47 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:11.962505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:11.980512 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69649cf2-b136-45a4-9e4b-1e6a12262142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:11.981331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69649cf2-b136-45a4-9e4b-1e6a12262142 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:11.981635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69649cf2-b136-45a4-9e4b-1e6a12262142 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:11.981907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69649cf2-b136-45a4-9e4b-1e6a12262142 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:11.990928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69649cf2-b136-45a4-9e4b-1e6a12262142 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:11.992443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69649cf2-b136-45a4-9e4b-1e6a12262142 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-69649cf2-b136-45a4-9e4b-1e6a12262142 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:36:12.008485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:12.009495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:12.010458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:12.010458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:12.010458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:36:12.010458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:36:12.010458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:36:12.010458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:36:12.010458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:36:12.010458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:36:12.010458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:36:12.010458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:12.010458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 463/972] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jul 19 20:36:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:12.016868 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb588b8-2ed5-4d18-9b08-ff15fd0c38e4 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:12.121651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb588b8-2ed5-4d18-9b08-ff15fd0c38e4 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:12.158149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e92dc-bc1a-452f-bd22-cfb7f03a514e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:12.158825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e92dc-bc1a-452f-bd22-cfb7f03a514e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:12.159110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e92dc-bc1a-452f-bd22-cfb7f03a514e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:12.159387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e92dc-bc1a-452f-bd22-cfb7f03a514e None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:12.160228 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e92dc-bc1a-452f-bd22-cfb7f03a514e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:12.160539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e92dc-bc1a-452f-bd22-cfb7f03a514e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:12.161987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e92dc-bc1a-452f-bd22-cfb7f03a514e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:12.180964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 510/973] 10.222.0.70 () {56 vars in 1149 bytes} [Tue Jul 19 20:36:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:12.186565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69649cf2-b136-45a4-9e4b-1e6a12262142 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:12.232105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ee213-c429-44bf-aeed-aea2c4a196ad None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:12.232547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ee213-c429-44bf-aeed-aea2c4a196ad None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:12.232832 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ee213-c429-44bf-aeed-aea2c4a196ad None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:12.234389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 511/974] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:36:12.239237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368754ae-abb5-414e-86f1-c0517fc3a432 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:12.239528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368754ae-abb5-414e-86f1-c0517fc3a432 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:12.239814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368754ae-abb5-414e-86f1-c0517fc3a432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:12.258241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-368754ae-abb5-414e-86f1-c0517fc3a432 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:12.259557 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-368754ae-abb5-414e-86f1-c0517fc3a432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:12.306459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:12.312536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69649cf2-b136-45a4-9e4b-1e6a12262142 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:12.333133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 512/975] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:36:12.352046 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb39357-5ff9-41f4-ab52-e74eef16327e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:12.353043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb39357-5ff9-41f4-ab52-e74eef16327e None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:12.353566 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb39357-5ff9-41f4-ab52-e74eef16327e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:12.354720 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb39357-5ff9-41f4-ab52-e74eef16327e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:12.356454 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368754ae-abb5-414e-86f1-c0517fc3a432 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:12.414396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:12.439387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb39357-5ff9-41f4-ab52-e74eef16327e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:12.440879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb39357-5ff9-41f4-ab52-e74eef16327e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:12.478050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368754ae-abb5-414e-86f1-c0517fc3a432 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:12.515699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5464b2d-cbf7-4535-b087-72fd7bbc6926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:12.516591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5464b2d-cbf7-4535-b087-72fd7bbc6926 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:12.516591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5464b2d-cbf7-4535-b087-72fd7bbc6926 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:12.516807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5464b2d-cbf7-4535-b087-72fd7bbc6926 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:12.517589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5464b2d-cbf7-4535-b087-72fd7bbc6926 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:12.518964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5464b2d-cbf7-4535-b087-72fd7bbc6926 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:12.533748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 513/976] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:36:12 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:36:12.535196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bf0c9-dfc0-4483-b9dd-f25c1f06b592 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:12.535464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bf0c9-dfc0-4483-b9dd-f25c1f06b592 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:12.535731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bf0c9-dfc0-4483-b9dd-f25c1f06b592 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:12.537390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 514/977] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:12.546591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c98fd-c993-4706-a5e0-3c20401ef2c9 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:12.546591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c98fd-c993-4706-a5e0-3c20401ef2c9 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:12.546591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c98fd-c993-4706-a5e0-3c20401ef2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:12.565819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-5d4c98fd-c993-4706-a5e0-3c20401ef2c9 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:12.567195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5d4c98fd-c993-4706-a5e0-3c20401ef2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:12.573553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 464/978] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:36:12 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:36:12.581521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb39357-5ff9-41f4-ab52-e74eef16327e None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:12.611881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:12.682607 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 515/979] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:12 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:12.714323 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:12.723288 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c98fd-c993-4706-a5e0-3c20401ef2c9 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:12.748819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb39357-5ff9-41f4-ab52-e74eef16327e None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:12.794938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4896973-7681-4d23-9807-4b9e5cc83b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:12.795755 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4896973-7681-4d23-9807-4b9e5cc83b42 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:12.796026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4896973-7681-4d23-9807-4b9e5cc83b42 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:12.796316 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4896973-7681-4d23-9807-4b9e5cc83b42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:12.807549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4896973-7681-4d23-9807-4b9e5cc83b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:12.809009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4896973-7681-4d23-9807-4b9e5cc83b42 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-e4896973-7681-4d23-9807-4b9e5cc83b42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:12.829103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:12.830612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:12.832561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:12.832561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:12.832561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:12.832561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:12.833513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 465/980] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:12.836625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4896973-7681-4d23-9807-4b9e5cc83b42 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:12.851899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c98fd-c993-4706-a5e0-3c20401ef2c9 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:12.890688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faad6ab-769e-41e5-97db-c05b75c73f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:12.891901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faad6ab-769e-41e5-97db-c05b75c73f1d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:12.892294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faad6ab-769e-41e5-97db-c05b75c73f1d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:12.892628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faad6ab-769e-41e5-97db-c05b75c73f1d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:12.893381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faad6ab-769e-41e5-97db-c05b75c73f1d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:12.894868 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faad6ab-769e-41e5-97db-c05b75c73f1d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-3faad6ab-769e-41e5-97db-c05b75c73f1d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:12.909694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:12.910766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:12.911805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 516/981] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jul 19 20:36:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:12.917076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faad6ab-769e-41e5-97db-c05b75c73f1d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:12.963850 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:12.980279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4896973-7681-4d23-9807-4b9e5cc83b42 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:13.026254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:13.033491 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acd6a9-86de-4c71-9b75-d3df0bc1bf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:13.034173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acd6a9-86de-4c71-9b75-d3df0bc1bf31 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:13.034459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acd6a9-86de-4c71-9b75-d3df0bc1bf31 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:13.034738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acd6a9-86de-4c71-9b75-d3df0bc1bf31 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:13.035478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acd6a9-86de-4c71-9b75-d3df0bc1bf31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:13.035790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acd6a9-86de-4c71-9b75-d3df0bc1bf31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:13.037163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acd6a9-86de-4c71-9b75-d3df0bc1bf31 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:13.049927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 466/982] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:13.055044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acd6a9-86de-4c71-9b75-d3df0bc1bf31 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:13.064287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faad6ab-769e-41e5-97db-c05b75c73f1d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:13.106349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80566f1-0a14-4373-ac63-8144cfdd4961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:13.107522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80566f1-0a14-4373-ac63-8144cfdd4961 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:13.107819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80566f1-0a14-4373-ac63-8144cfdd4961 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:13.108259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80566f1-0a14-4373-ac63-8144cfdd4961 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:13.109061 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80566f1-0a14-4373-ac63-8144cfdd4961 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:13.109475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80566f1-0a14-4373-ac63-8144cfdd4961 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:13.110926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80566f1-0a14-4373-ac63-8144cfdd4961 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:13.117960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 517/983] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jul 19 20:36:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:13.124464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80566f1-0a14-4373-ac63-8144cfdd4961 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:13.179961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acd6a9-86de-4c71-9b75-d3df0bc1bf31 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:13.244197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0529c2f-ae3b-464c-8e30-ccbe18f3cc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:13.245871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529c2f-ae3b-464c-8e30-ccbe18f3cc83 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:13.246329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529c2f-ae3b-464c-8e30-ccbe18f3cc83 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:13.246797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529c2f-ae3b-464c-8e30-ccbe18f3cc83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:13.258885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0529c2f-ae3b-464c-8e30-ccbe18f3cc83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:13.261733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0529c2f-ae3b-464c-8e30-ccbe18f3cc83 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:13.268069 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:13.284068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80566f1-0a14-4373-ac63-8144cfdd4961 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-c0529c2f-ae3b-464c-8e30-ccbe18f3cc83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:36:13.286676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:13.288656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:13.290065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:36:13.290065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:36:13.290065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:36:13.290065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:36:13.290065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:36:13.290065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:36:13.290065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:13.290065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:13.290065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 467/984] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:13.298548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0529c2f-ae3b-464c-8e30-ccbe18f3cc83 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:13.355454 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0f059-28f3-4129-afcd-19d99271af6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:13.356584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f059-28f3-4129-afcd-19d99271af6f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:13.357033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f059-28f3-4129-afcd-19d99271af6f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:13.357520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f059-28f3-4129-afcd-19d99271af6f None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:13.358666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0f059-28f3-4129-afcd-19d99271af6f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:13.359176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0f059-28f3-4129-afcd-19d99271af6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:13.360964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0f059-28f3-4129-afcd-19d99271af6f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:13.371577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 518/985] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jul 19 20:36:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:13.377722 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0f059-28f3-4129-afcd-19d99271af6f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:13.392770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:13.483828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0529c2f-ae3b-464c-8e30-ccbe18f3cc83 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:13.538428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af064-54eb-4f63-9076-ae154f7fbffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:13.539481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af064-54eb-4f63-9076-ae154f7fbffc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:13.539924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af064-54eb-4f63-9076-ae154f7fbffc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:13.540303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af064-54eb-4f63-9076-ae154f7fbffc None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:13.541389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af064-54eb-4f63-9076-ae154f7fbffc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:13.542223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af064-54eb-4f63-9076-ae154f7fbffc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:13.542398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0f059-28f3-4129-afcd-19d99271af6f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:13.544182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af064-54eb-4f63-9076-ae154f7fbffc None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:13.573028 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 468/986] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:36:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:13.574731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af064-54eb-4f63-9076-ae154f7fbffc None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:13.589985 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c4a18-83b7-4360-a676-79236bc3c986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:13.590742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c4a18-83b7-4360-a676-79236bc3c986 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:13.591142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c4a18-83b7-4360-a676-79236bc3c986 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:13.591770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c4a18-83b7-4360-a676-79236bc3c986 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:13.592820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c4a18-83b7-4360-a676-79236bc3c986 None admin] RBAC: Adding query filter params (interface=admin, service_id=fe768e6693574f05a864de3b9789557c, region_id=RegionOne) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:13.593147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c4a18-83b7-4360-a676-79236bc3c986 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:13.594628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c4a18-83b7-4360-a676-79236bc3c986 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:13.605422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 519/987] 10.222.0.70 () {56 vars in 1249 bytes} [Tue Jul 19 20:36:13 2022] GET /identity/v3/endpoints?service_id=fe768e6693574f05a864de3b9789557c&interface=admin®ion_id=RegionOne => generated 193 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:13.609025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c4a18-83b7-4360-a676-79236bc3c986 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:13.641846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:13.735776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af064-54eb-4f63-9076-ae154f7fbffc None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:13.749598 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:13.757534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c4a18-83b7-4360-a676-79236bc3c986 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:13.786629 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c7d3d-2f18-4be1-a230-b639aa7c9fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:13.787831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c7d3d-2f18-4be1-a230-b639aa7c9fb0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:13.787831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c7d3d-2f18-4be1-a230-b639aa7c9fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:13.788035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c7d3d-2f18-4be1-a230-b639aa7c9fb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:13.789086 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c7d3d-2f18-4be1-a230-b639aa7c9fb0 None admin] RBAC: Adding query filter params (role.id=7833471323564c3794e65a6b05374aa4, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=2711df504d424fa0b05cd16d7fd5acfb) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:13.789338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c7d3d-2f18-4be1-a230-b639aa7c9fb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:13.791320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c7d3d-2f18-4be1-a230-b639aa7c9fb0 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:13.809401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 469/988] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:36:13 2022] GET /identity/v3/role_assignments?user.id=2711df504d424fa0b05cd16d7fd5acfb&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=7833471323564c3794e65a6b05374aa4 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:13.809631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b0bfe-6e09-42ea-98ca-8297a9ff94c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:13.809631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b0bfe-6e09-42ea-98ca-8297a9ff94c2 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:13.809631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b0bfe-6e09-42ea-98ca-8297a9ff94c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:13.809631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b0bfe-6e09-42ea-98ca-8297a9ff94c2 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:13.809977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b0bfe-6e09-42ea-98ca-8297a9ff94c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:13.809977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b0bfe-6e09-42ea-98ca-8297a9ff94c2 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:13.820501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 520/989] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jul 19 20:36:13 2022] GET /identity/v3/services => generated 1727 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:36:14.861011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2adf11-d071-4e9d-8cda-660ec2837096 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:14.861862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2adf11-d071-4e9d-8cda-660ec2837096 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:14.862011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2adf11-d071-4e9d-8cda-660ec2837096 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:14.863453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 470/990] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:14.870304 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4943198-9e6f-469c-b6d0-736136bdcff9 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:14.870834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4943198-9e6f-469c-b6d0-736136bdcff9 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:14.871409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4943198-9e6f-469c-b6d0-736136bdcff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:14.901910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f4943198-9e6f-469c-b6d0-736136bdcff9 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:14.903935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f4943198-9e6f-469c-b6d0-736136bdcff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:15.031123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 521/991] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:15.054907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2adf11-d071-4e9d-8cda-660ec2837096 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:15.206753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2adf11-d071-4e9d-8cda-660ec2837096 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:15.255424 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960e151-a8e9-49dd-a5af-71ec7f57fc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:15.256702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960e151-a8e9-49dd-a5af-71ec7f57fc8c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:15.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960e151-a8e9-49dd-a5af-71ec7f57fc8c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:15.258673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960e151-a8e9-49dd-a5af-71ec7f57fc8c None admin] PATH_INFO: `/v3/services/compute` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:15.258673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960e151-a8e9-49dd-a5af-71ec7f57fc8c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:15.260766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960e151-a8e9-49dd-a5af-71ec7f57fc8c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-6960e151-a8e9-49dd-a5af-71ec7f57fc8c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:15.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:15.281477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:36:15.283851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:15.286184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 471/992] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:36:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:15.290529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4943198-9e6f-469c-b6d0-736136bdcff9 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:15.321130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:15.478740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4943198-9e6f-469c-b6d0-736136bdcff9 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:15.532071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd4fe1-619d-484a-bc01-9643cc14ad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:15.533385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd4fe1-619d-484a-bc01-9643cc14ad4b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:15.533970 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd4fe1-619d-484a-bc01-9643cc14ad4b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:15.534550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd4fe1-619d-484a-bc01-9643cc14ad4b None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:15.535878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd4fe1-619d-484a-bc01-9643cc14ad4b None admin] RBAC: Adding query filter params (name=compute) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:15.536503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd4fe1-619d-484a-bc01-9643cc14ad4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:15.538594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd4fe1-619d-484a-bc01-9643cc14ad4b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:15.547127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 522/993] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:36:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:15.557322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960e151-a8e9-49dd-a5af-71ec7f57fc8c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:15.613206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:15.698100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960e151-a8e9-49dd-a5af-71ec7f57fc8c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:15.743344 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c9251-3512-4efd-be45-5eae296ce72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:15.743969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c9251-3512-4efd-be45-5eae296ce72b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:15.744252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c9251-3512-4efd-be45-5eae296ce72b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:15.744531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c9251-3512-4efd-be45-5eae296ce72b None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:15.745960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c9251-3512-4efd-be45-5eae296ce72b None admin] RBAC: Adding query filter params (type=compute) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:15.746313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c9251-3512-4efd-be45-5eae296ce72b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:15.747859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c9251-3512-4efd-be45-5eae296ce72b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:15.754446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 472/994] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:36:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:15.760358 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd4fe1-619d-484a-bc01-9643cc14ad4b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:15.907070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:15.960278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd4fe1-619d-484a-bc01-9643cc14ad4b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:16.032570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a4848-af51-480c-b919-c2df9a02241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:16.033708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a4848-af51-480c-b919-c2df9a02241a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:16.034236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a4848-af51-480c-b919-c2df9a02241a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:16.034751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a4848-af51-480c-b919-c2df9a02241a None admin] PATH_INFO: `/v3/endpoints` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:16.036428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a4848-af51-480c-b919-c2df9a02241a None admin] RBAC: Adding query filter params (interface=public, service_id=d0cb6a28c7ae41a3ba659bc1c4db5a4d, region_id=RegionOne) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:16.037008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a4848-af51-480c-b919-c2df9a02241a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:16.039701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a4848-af51-480c-b919-c2df9a02241a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:16.054220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 523/995] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jul 19 20:36:15 2022] GET /identity/v3/endpoints?service_id=d0cb6a28c7ae41a3ba659bc1c4db5a4d&interface=public®ion_id=RegionOne => generated 194 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:16.055664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c9251-3512-4efd-be45-5eae296ce72b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:16.083360 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee8768-19d1-42b4-abd4-dce0cfd42236 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:16.083709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee8768-19d1-42b4-abd4-dce0cfd42236 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:16.083936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee8768-19d1-42b4-abd4-dce0cfd42236 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:16.085631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 524/996] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:36:16.092051 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da50bf-ee0c-4d9e-b18f-ff95becf8eb9 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:16.092345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da50bf-ee0c-4d9e-b18f-ff95becf8eb9 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:16.092625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da50bf-ee0c-4d9e-b18f-ff95becf8eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:16.113222 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-d5da50bf-ee0c-4d9e-b18f-ff95becf8eb9 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:16.114552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d5da50bf-ee0c-4d9e-b18f-ff95becf8eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:16.199334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 525/997] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:16.231037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da50bf-ee0c-4d9e-b18f-ff95becf8eb9 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:16.232443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c9251-3512-4efd-be45-5eae296ce72b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:16.235186 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:16.303276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c3362-cc0a-4303-b85e-c8a4815aaff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:16.304099 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c3362-cc0a-4303-b85e-c8a4815aaff3 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:16.304388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c3362-cc0a-4303-b85e-c8a4815aaff3 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:16.304788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c3362-cc0a-4303-b85e-c8a4815aaff3 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:16.305739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c3362-cc0a-4303-b85e-c8a4815aaff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:16.308071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c3362-cc0a-4303-b85e-c8a4815aaff3 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:16.317794 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 473/998] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jul 19 20:36:16 2022] GET /identity/v3/services => generated 1727 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:36:16.392394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:16.429495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da50bf-ee0c-4d9e-b18f-ff95becf8eb9 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:16.430446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc615c-3b10-43ed-af20-8efedea58574 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:16.430958 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc615c-3b10-43ed-af20-8efedea58574 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:16.431698 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc615c-3b10-43ed-af20-8efedea58574 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:16.435147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 474/999] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:16.441421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547973f5-25d4-483d-8712-ef9faba5b4de None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:16.441623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547973f5-25d4-483d-8712-ef9faba5b4de None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:16.441843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547973f5-25d4-483d-8712-ef9faba5b4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:16.462386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-547973f5-25d4-483d-8712-ef9faba5b4de None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:16.462591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-547973f5-25d4-483d-8712-ef9faba5b4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:16.476795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00a6ce-0da6-4a3c-8aa5-c39f25101aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:16.477634 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00a6ce-0da6-4a3c-8aa5-c39f25101aee None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:16.477980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00a6ce-0da6-4a3c-8aa5-c39f25101aee None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:16.478261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00a6ce-0da6-4a3c-8aa5-c39f25101aee None admin] PATH_INFO: `/v3/services/image` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:16.479472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00a6ce-0da6-4a3c-8aa5-c39f25101aee None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:16.482024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00a6ce-0da6-4a3c-8aa5-c39f25101aee None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-ba00a6ce-0da6-4a3c-8aa5-c39f25101aee None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:16.495644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:16.496705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:36:16.497644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:16.501448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 526/1000] 10.222.0.70 () {56 vars in 1105 bytes} [Tue Jul 19 20:36:16 2022] GET /identity/v3/services/image => generated 86 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:16.503499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00a6ce-0da6-4a3c-8aa5-c39f25101aee None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:16.547726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 475/1001] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:36:16.572602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547973f5-25d4-483d-8712-ef9faba5b4de None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:16.657039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00a6ce-0da6-4a3c-8aa5-c39f25101aee None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:16.711108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:16.712405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3c8c8-d1ac-4643-966d-ae7e6c1288a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:16.713604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3c8c8-d1ac-4643-966d-ae7e6c1288a3 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:16.714159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3c8c8-d1ac-4643-966d-ae7e6c1288a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:16.714677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3c8c8-d1ac-4643-966d-ae7e6c1288a3 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:16.715742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3c8c8-d1ac-4643-966d-ae7e6c1288a3 None admin] RBAC: Adding query filter params (name=image) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:16.716263 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3c8c8-d1ac-4643-966d-ae7e6c1288a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:16.717755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3c8c8-d1ac-4643-966d-ae7e6c1288a3 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:16.718600 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547973f5-25d4-483d-8712-ef9faba5b4de None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:16.721193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:16.726679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 527/1002] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:16.732152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3c8c8-d1ac-4643-966d-ae7e6c1288a3 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:16.768723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff492d1d-d7a1-4d79-b32b-668c2e0fcf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:16.769952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff492d1d-d7a1-4d79-b32b-668c2e0fcf02 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:16.770493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff492d1d-d7a1-4d79-b32b-668c2e0fcf02 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:16.770950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff492d1d-d7a1-4d79-b32b-668c2e0fcf02 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:16.771851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff492d1d-d7a1-4d79-b32b-668c2e0fcf02 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:16.773991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff492d1d-d7a1-4d79-b32b-668c2e0fcf02 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-ff492d1d-d7a1-4d79-b32b-668c2e0fcf02 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:16.787841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:16.789106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:36:16.790156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:16.797406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 476/1003] 10.222.0.70 () {56 vars in 1126 bytes} [Tue Jul 19 20:36:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:16.798832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff492d1d-d7a1-4d79-b32b-668c2e0fcf02 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:16.844254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3c8c8-d1ac-4643-966d-ae7e6c1288a3 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:16.880525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47143e-2c14-4239-9b13-01261f1defad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:16.880991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47143e-2c14-4239-9b13-01261f1defad None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:16.881320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47143e-2c14-4239-9b13-01261f1defad None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:16.881853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47143e-2c14-4239-9b13-01261f1defad None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:16.882882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47143e-2c14-4239-9b13-01261f1defad None admin] RBAC: Adding query filter params (type=image) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:16.883216 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47143e-2c14-4239-9b13-01261f1defad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:16.884986 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47143e-2c14-4239-9b13-01261f1defad None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:16.890457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 528/1004] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:16.910904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff492d1d-d7a1-4d79-b32b-668c2e0fcf02 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:16.962083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b68d7-5c63-4a6e-9d4f-f3637b787bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:16.962768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b68d7-5c63-4a6e-9d4f-f3637b787bca None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:16.963053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b68d7-5c63-4a6e-9d4f-f3637b787bca None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:16.963333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b68d7-5c63-4a6e-9d4f-f3637b787bca None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:16.964074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b68d7-5c63-4a6e-9d4f-f3637b787bca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:16.964386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b68d7-5c63-4a6e-9d4f-f3637b787bca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:16.965972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b68d7-5c63-4a6e-9d4f-f3637b787bca None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:16.976581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 477/1005] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jul 19 20:36:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:16.981989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47143e-2c14-4239-9b13-01261f1defad None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:17.058543 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:17.100973 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47143e-2c14-4239-9b13-01261f1defad None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:17.145902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d67eb-41ca-41c7-95ef-83d1cd479fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:17.146580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d67eb-41ca-41c7-95ef-83d1cd479fd8 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:17.146866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d67eb-41ca-41c7-95ef-83d1cd479fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:17.147144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d67eb-41ca-41c7-95ef-83d1cd479fd8 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:17.147880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d67eb-41ca-41c7-95ef-83d1cd479fd8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:17.148193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d67eb-41ca-41c7-95ef-83d1cd479fd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:17.149585 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d67eb-41ca-41c7-95ef-83d1cd479fd8 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:17.159778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 529/1006] 10.222.0.70 () {56 vars in 1122 bytes} [Tue Jul 19 20:36:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:17.161043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b68d7-5c63-4a6e-9d4f-f3637b787bca None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:17.193683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:17.278543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b68d7-5c63-4a6e-9d4f-f3637b787bca None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:17.312104 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b375f72-64bc-4fa8-8f3f-caef745ee1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:17.312792 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b375f72-64bc-4fa8-8f3f-caef745ee1b2 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:17.313079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b375f72-64bc-4fa8-8f3f-caef745ee1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:17.313384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b375f72-64bc-4fa8-8f3f-caef745ee1b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:17.314305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b375f72-64bc-4fa8-8f3f-caef745ee1b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:17.315600 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b375f72-64bc-4fa8-8f3f-caef745ee1b2 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:17.341395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 478/1007] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jul 19 20:36:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:36:18.519811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7477945-270e-4645-95af-4f2323780bc2 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:18.520396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7477945-270e-4645-95af-4f2323780bc2 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:18.520396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7477945-270e-4645-95af-4f2323780bc2 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:18.521822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 530/1008] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:18.527703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a373d0-1612-41f8-ac1e-09a22f370dbf None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:18.528160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a373d0-1612-41f8-ac1e-09a22f370dbf None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:18.528330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a373d0-1612-41f8-ac1e-09a22f370dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:18.545435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c6a373d0-1612-41f8-ac1e-09a22f370dbf None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:18.546646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c6a373d0-1612-41f8-ac1e-09a22f370dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:18.622296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 479/1009] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:18 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:18.646460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7477945-270e-4645-95af-4f2323780bc2 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:18.750237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7477945-270e-4645-95af-4f2323780bc2 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:18.778331 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:18.787996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb563a-24f7-4897-b0e5-aed4e3dc0a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:18.788807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb563a-24f7-4897-b0e5-aed4e3dc0a77 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:18.789093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb563a-24f7-4897-b0e5-aed4e3dc0a77 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:18.789387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb563a-24f7-4897-b0e5-aed4e3dc0a77 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:18.790050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb563a-24f7-4897-b0e5-aed4e3dc0a77 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:18.791437 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb563a-24f7-4897-b0e5-aed4e3dc0a77 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-eedb563a-24f7-4897-b0e5-aed4e3dc0a77 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:18.803425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:18.804599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:18.805560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 531/1010] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:36:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:18.812768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a373d0-1612-41f8-ac1e-09a22f370dbf None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:18.826315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:18.919906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a373d0-1612-41f8-ac1e-09a22f370dbf None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:18.958640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75b79a-b76a-4a1d-90e8-7655a1f6868f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:18.959324 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75b79a-b76a-4a1d-90e8-7655a1f6868f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:18.959609 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75b79a-b76a-4a1d-90e8-7655a1f6868f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:18.959895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75b79a-b76a-4a1d-90e8-7655a1f6868f None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:18.960617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75b79a-b76a-4a1d-90e8-7655a1f6868f None admin] RBAC: Adding query filter params (name=compute) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:18.960934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75b79a-b76a-4a1d-90e8-7655a1f6868f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:18.962361 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75b79a-b76a-4a1d-90e8-7655a1f6868f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:18.974486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 480/1011] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:36:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:18.974707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb563a-24f7-4897-b0e5-aed4e3dc0a77 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:19.000391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0744a-948c-4034-82b1-4ebc11e25772 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:19.000735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0744a-948c-4034-82b1-4ebc11e25772 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:19.000954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0744a-948c-4034-82b1-4ebc11e25772 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:19.002663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 481/1012] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:36:19.007728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b77b1-77de-409e-bb33-5dabcc300cf4 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:19.008054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b77b1-77de-409e-bb33-5dabcc300cf4 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:19.008273 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b77b1-77de-409e-bb33-5dabcc300cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:19.027476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-8e1b77b1-77de-409e-bb33-5dabcc300cf4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:19.029957 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8e1b77b1-77de-409e-bb33-5dabcc300cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:19.088474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb563a-24f7-4897-b0e5-aed4e3dc0a77 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:19.117381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 482/1013] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:36:19.123673 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297e0e9-eb44-48be-8a8e-15f7be632c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:19.124348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297e0e9-eb44-48be-8a8e-15f7be632c4e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:19.124636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297e0e9-eb44-48be-8a8e-15f7be632c4e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:19.124965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297e0e9-eb44-48be-8a8e-15f7be632c4e None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:19.125775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297e0e9-eb44-48be-8a8e-15f7be632c4e None admin] RBAC: Adding query filter params (type=compute) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:19.126025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297e0e9-eb44-48be-8a8e-15f7be632c4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:19.127391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297e0e9-eb44-48be-8a8e-15f7be632c4e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:19.133420 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b77b1-77de-409e-bb33-5dabcc300cf4 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:19.134164 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 532/1014] 10.222.0.70 () {56 vars in 1112 bytes} [Tue Jul 19 20:36:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:19.137749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:19.141849 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297e0e9-eb44-48be-8a8e-15f7be632c4e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:19.205987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:19.246242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b77b1-77de-409e-bb33-5dabcc300cf4 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:19.254380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297e0e9-eb44-48be-8a8e-15f7be632c4e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:19.285379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82d736-69c0-4e4d-aeab-28dc86e6e1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:19.286414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d736-69c0-4e4d-aeab-28dc86e6e1e7 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:19.286699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d736-69c0-4e4d-aeab-28dc86e6e1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:19.286989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d736-69c0-4e4d-aeab-28dc86e6e1e7 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:19.287768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82d736-69c0-4e4d-aeab-28dc86e6e1e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:19.289137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82d736-69c0-4e4d-aeab-28dc86e6e1e7 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:19.291139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285a05c-4570-45d2-bf18-1a2e80749842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:19.292342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a05c-4570-45d2-bf18-1a2e80749842 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:19.292631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a05c-4570-45d2-bf18-1a2e80749842 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:19.292916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a05c-4570-45d2-bf18-1a2e80749842 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:19.293694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285a05c-4570-45d2-bf18-1a2e80749842 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:19.295198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285a05c-4570-45d2-bf18-1a2e80749842 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:19.304599 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 483/1015] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:36:19 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:36:19.329399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 533/1016] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jul 19 20:36:19 2022] POST /identity/v3/endpoints => generated 334 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:36:19.444045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d4fcc-b6cb-4519-aa97-4395ef6634f7 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:19.444354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d4fcc-b6cb-4519-aa97-4395ef6634f7 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:19.444733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d4fcc-b6cb-4519-aa97-4395ef6634f7 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:19.446391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 484/1017] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:19.452520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab7222-c2e3-456c-a621-246e2dba5e94 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:19.452864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab7222-c2e3-456c-a621-246e2dba5e94 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:19.453079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab7222-c2e3-456c-a621-246e2dba5e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:19.472176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-38ab7222-c2e3-456c-a621-246e2dba5e94 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:19.473350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-38ab7222-c2e3-456c-a621-246e2dba5e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:19.555103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 534/1018] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:19.579356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d4fcc-b6cb-4519-aa97-4395ef6634f7 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:19.651268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:19.694266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d4fcc-b6cb-4519-aa97-4395ef6634f7 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:19.730448 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f114d6-4e0f-41e9-a035-f15671a14044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:19.731157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f114d6-4e0f-41e9-a035-f15671a14044 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:19.731439 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f114d6-4e0f-41e9-a035-f15671a14044 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:19.731714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f114d6-4e0f-41e9-a035-f15671a14044 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:19.732891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f114d6-4e0f-41e9-a035-f15671a14044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:19.734481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f114d6-4e0f-41e9-a035-f15671a14044 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:19.740823 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 485/1019] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:19.742345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab7222-c2e3-456c-a621-246e2dba5e94 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:19.788751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:19.875400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab7222-c2e3-456c-a621-246e2dba5e94 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:19.914224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf4c52-e9a8-47fc-af94-621a390f5dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:19.914902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf4c52-e9a8-47fc-af94-621a390f5dcc None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:19.915189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf4c52-e9a8-47fc-af94-621a390f5dcc None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:19.915462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf4c52-e9a8-47fc-af94-621a390f5dcc None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:19.916192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf4c52-e9a8-47fc-af94-621a390f5dcc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:19.918258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf4c52-e9a8-47fc-af94-621a390f5dcc None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:19.945547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 535/1020] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:36:19 2022] POST /identity/v3/projects => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:36:21.476702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b4684-5d6b-4b58-a8b2-aeecd5212874 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:21.477227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b4684-5d6b-4b58-a8b2-aeecd5212874 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:21.477227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b4684-5d6b-4b58-a8b2-aeecd5212874 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:21.478819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 486/1021] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:21.483827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e26ad4-db85-4947-832a-5d566c40f52c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:21.484250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e26ad4-db85-4947-832a-5d566c40f52c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:21.484445 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e26ad4-db85-4947-832a-5d566c40f52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:21.502027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-93e26ad4-db85-4947-832a-5d566c40f52c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:21.503268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-93e26ad4-db85-4947-832a-5d566c40f52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:21.519611 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85796a8d-dae8-44d7-a8a4-2ebfb7c0e3bb None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:21.519914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85796a8d-dae8-44d7-a8a4-2ebfb7c0e3bb None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:21.520195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85796a8d-dae8-44d7-a8a4-2ebfb7c0e3bb None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:21.525394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 487/1022] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:36:21.530090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bd7f5-aded-4445-b28c-675058936e0a None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:21.530648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bd7f5-aded-4445-b28c-675058936e0a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:21.530924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bd7f5-aded-4445-b28c-675058936e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:21.551610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-541bd7f5-aded-4445-b28c-675058936e0a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:21.553556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-541bd7f5-aded-4445-b28c-675058936e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:21.593595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 536/1023] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:21.615053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e26ad4-db85-4947-832a-5d566c40f52c None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:21.644190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:21.644670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 488/1024] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:21.669204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bd7f5-aded-4445-b28c-675058936e0a None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:21.762943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:21.769858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e26ad4-db85-4947-832a-5d566c40f52c None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:21.808370 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaec7f-0e33-4ef2-aa19-57e0c0482e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:21.809165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaec7f-0e33-4ef2-aa19-57e0c0482e42 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:21.809335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bd7f5-aded-4445-b28c-675058936e0a None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:21.809500 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaec7f-0e33-4ef2-aa19-57e0c0482e42 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:21.809736 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaec7f-0e33-4ef2-aa19-57e0c0482e42 None admin] PATH_INFO: `/v3/services/image` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:21.810392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aaec7f-0e33-4ef2-aa19-57e0c0482e42 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:21.812348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aaec7f-0e33-4ef2-aa19-57e0c0482e42 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-b8aaec7f-0e33-4ef2-aa19-57e0c0482e42 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:21.823886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:21.825251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:21.826155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 537/1025] 10.222.0.70 () {56 vars in 1105 bytes} [Tue Jul 19 20:36:21 2022] GET /identity/v3/services/image => generated 86 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:21.832296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaec7f-0e33-4ef2-aa19-57e0c0482e42 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:21.849834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c13ca-e04b-475d-a7cf-92671ed9395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:21.851073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c13ca-e04b-475d-a7cf-92671ed9395d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:21.851517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c13ca-e04b-475d-a7cf-92671ed9395d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:21.851800 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c13ca-e04b-475d-a7cf-92671ed9395d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:21.863369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c13ca-e04b-475d-a7cf-92671ed9395d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-302c13ca-e04b-475d-a7cf-92671ed9395d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:36:21.872278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:21.873428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:21.874306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 489/1026] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jul 19 20:36:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:21.879531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c13ca-e04b-475d-a7cf-92671ed9395d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:21.935059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:21.955312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaec7f-0e33-4ef2-aa19-57e0c0482e42 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:22.007249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c13ca-e04b-475d-a7cf-92671ed9395d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:22.008434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfbb82-7118-40bf-a341-98b983cba115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:22.009114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfbb82-7118-40bf-a341-98b983cba115 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.009422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfbb82-7118-40bf-a341-98b983cba115 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.009702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfbb82-7118-40bf-a341-98b983cba115 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.010430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfbb82-7118-40bf-a341-98b983cba115 None admin] RBAC: Adding query filter params (name=image) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:22.010741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfbb82-7118-40bf-a341-98b983cba115 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:22.012107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfbb82-7118-40bf-a341-98b983cba115 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:22.017713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 538/1027] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:22.024116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfbb82-7118-40bf-a341-98b983cba115 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:22.038680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:22.053942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642a0e0-f9ce-4a88-ac65-a4cb3979a2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:22.054626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642a0e0-f9ce-4a88-ac65-a4cb3979a2b0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.054907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642a0e0-f9ce-4a88-ac65-a4cb3979a2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.055185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642a0e0-f9ce-4a88-ac65-a4cb3979a2b0 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.056524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642a0e0-f9ce-4a88-ac65-a4cb3979a2b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:22.057066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642a0e0-f9ce-4a88-ac65-a4cb3979a2b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:22.059619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642a0e0-f9ce-4a88-ac65-a4cb3979a2b0 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:22.070240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 490/1028] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jul 19 20:36:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:22.076360 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642a0e0-f9ce-4a88-ac65-a4cb3979a2b0 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:22.138961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfbb82-7118-40bf-a341-98b983cba115 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:22.174237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454dec40-47bc-4528-9c27-abbeead64b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:22.174903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dec40-47bc-4528-9c27-abbeead64b14 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.175291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dec40-47bc-4528-9c27-abbeead64b14 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.175462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dec40-47bc-4528-9c27-abbeead64b14 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.176180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454dec40-47bc-4528-9c27-abbeead64b14 None admin] RBAC: Adding query filter params (type=image) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:22.176485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454dec40-47bc-4528-9c27-abbeead64b14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:22.177896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454dec40-47bc-4528-9c27-abbeead64b14 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:22.182942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 539/1029] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:22.184193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230f1d-d8bf-4e71-afd6-2e03cb4940a4 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.184299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642a0e0-f9ce-4a88-ac65-a4cb3979a2b0 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:22.184481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230f1d-d8bf-4e71-afd6-2e03cb4940a4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.184757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230f1d-d8bf-4e71-afd6-2e03cb4940a4 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.189472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 540/1030] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:22.189472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4230f1d-d8bf-4e71-afd6-2e03cb4940a4 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:22.204838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:22.222818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29617e0-872d-444d-b711-873c4f95e8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:22.223616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29617e0-872d-444d-b711-873c4f95e8e8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.223894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29617e0-872d-444d-b711-873c4f95e8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.224164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29617e0-872d-444d-b711-873c4f95e8e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.236232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29617e0-872d-444d-b711-873c4f95e8e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:22.238316 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29617e0-872d-444d-b711-873c4f95e8e8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-d29617e0-872d-444d-b711-873c4f95e8e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:22.256995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:22.258047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:22.259052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:22.259052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:22.259052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:22.259052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:22.259052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 491/1031] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:22.259606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49d3f7-9a2c-442d-918a-a520a53114fa None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.259897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49d3f7-9a2c-442d-918a-a520a53114fa None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.260754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49d3f7-9a2c-442d-918a-a520a53114fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.278222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ce49d3f7-9a2c-442d-918a-a520a53114fa None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:22.279456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ce49d3f7-9a2c-442d-918a-a520a53114fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:22.305001 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4230f1d-d8bf-4e71-afd6-2e03cb4940a4 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:22.310324 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:22.352094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624a6f0-f188-46d5-adc0-ed43359db3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:22.352336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 492/1032] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:22.352598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624a6f0-f188-46d5-adc0-ed43359db3d1 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.352994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624a6f0-f188-46d5-adc0-ed43359db3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.353088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624a6f0-f188-46d5-adc0-ed43359db3d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.353161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49d3f7-9a2c-442d-918a-a520a53114fa None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:22.354349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624a6f0-f188-46d5-adc0-ed43359db3d1 None admin] RBAC: Adding query filter params (interface=public, service_id=38900ec4054849e39b75ac7afcf2de1e, region_id=RegionOne) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:22.354664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624a6f0-f188-46d5-adc0-ed43359db3d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:22.356010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624a6f0-f188-46d5-adc0-ed43359db3d1 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:22.362385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 541/1033] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jul 19 20:36:22 2022] GET /identity/v3/endpoints?service_id=38900ec4054849e39b75ac7afcf2de1e&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:22.367771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624a6f0-f188-46d5-adc0-ed43359db3d1 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:22.476484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49d3f7-9a2c-442d-918a-a520a53114fa None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:22.481025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:22.518330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624a6f0-f188-46d5-adc0-ed43359db3d1 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:22.519015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b7e8b-3d8e-4cf5-a280-ee88960d5e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:22.519396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b7e8b-3d8e-4cf5-a280-ee88960d5e18 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.519642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b7e8b-3d8e-4cf5-a280-ee88960d5e18 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.519946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b7e8b-3d8e-4cf5-a280-ee88960d5e18 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.520861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b7e8b-3d8e-4cf5-a280-ee88960d5e18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:22.521008 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b7e8b-3d8e-4cf5-a280-ee88960d5e18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:22.522509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b7e8b-3d8e-4cf5-a280-ee88960d5e18 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:22.534102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 493/1034] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:22.534938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b7e8b-3d8e-4cf5-a280-ee88960d5e18 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:22.564156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:22.569150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a61eba-ada9-4818-8794-e7922b6ae513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:22.569876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a61eba-ada9-4818-8794-e7922b6ae513 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.570164 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a61eba-ada9-4818-8794-e7922b6ae513 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.570443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a61eba-ada9-4818-8794-e7922b6ae513 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.571135 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a61eba-ada9-4818-8794-e7922b6ae513 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:22.572547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a61eba-ada9-4818-8794-e7922b6ae513 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:22.579916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 542/1035] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jul 19 20:36:22 2022] GET /identity/v3/services => generated 1957 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:36:22.580753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a61eba-ada9-4818-8794-e7922b6ae513 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:22.662450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b7e8b-3d8e-4cf5-a280-ee88960d5e18 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:22.712202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2cde8-5949-445d-b612-011cb57405b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:22.713021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2cde8-5949-445d-b612-011cb57405b3 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.713877 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2cde8-5949-445d-b612-011cb57405b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.714114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2cde8-5949-445d-b612-011cb57405b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.715412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2cde8-5949-445d-b612-011cb57405b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:22.716913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2cde8-5949-445d-b612-011cb57405b3 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:22.718858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 494/1036] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:22.725040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2cde8-5949-445d-b612-011cb57405b3 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:22.745059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a61eba-ada9-4818-8794-e7922b6ae513 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:22.791094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839e77e-d06b-4de0-9609-6d801e2b7dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:22.792478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839e77e-d06b-4de0-9609-6d801e2b7dd9 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.792769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839e77e-d06b-4de0-9609-6d801e2b7dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.793051 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839e77e-d06b-4de0-9609-6d801e2b7dd9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.802172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839e77e-d06b-4de0-9609-6d801e2b7dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:22.803644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839e77e-d06b-4de0-9609-6d801e2b7dd9 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:22.806611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-d839e77e-d06b-4de0-9609-6d801e2b7dd9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:36:22.821396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:22.823141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:22.825289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:22.825289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:22.825289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:36:22.825289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:36:22.825289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:36:22.825289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:36:22.825289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:36:22.825289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:36:22.825289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:36:22.825289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:22.825289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 543/1037] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jul 19 20:36:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:22.831522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839e77e-d06b-4de0-9609-6d801e2b7dd9 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:22.850147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:22.865178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2cde8-5949-445d-b612-011cb57405b3 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:22.919893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8927e95-7242-4b95-a061-78a9e1ed60dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:22.920590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8927e95-7242-4b95-a061-78a9e1ed60dd None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:22.920877 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8927e95-7242-4b95-a061-78a9e1ed60dd None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:22.921162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8927e95-7242-4b95-a061-78a9e1ed60dd None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:22.922016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8927e95-7242-4b95-a061-78a9e1ed60dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:22.923481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8927e95-7242-4b95-a061-78a9e1ed60dd None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:22.961754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 495/1038] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:36:22 2022] POST /identity/v3/users => generated 285 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:36:22.992283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839e77e-d06b-4de0-9609-6d801e2b7dd9 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:23.060070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f64f79-93ea-45de-a54c-3cc1398596ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:23.061239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f64f79-93ea-45de-a54c-3cc1398596ad None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:23.061776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f64f79-93ea-45de-a54c-3cc1398596ad None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:23.062194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f64f79-93ea-45de-a54c-3cc1398596ad None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:23.063574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f64f79-93ea-45de-a54c-3cc1398596ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:23.064135 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f64f79-93ea-45de-a54c-3cc1398596ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:23.066825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f64f79-93ea-45de-a54c-3cc1398596ad None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:23.108694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 544/1039] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jul 19 20:36:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:23.110354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8927e95-7242-4b95-a061-78a9e1ed60dd None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:23.249643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8927e95-7242-4b95-a061-78a9e1ed60dd None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:23.292459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857e384-45d9-448b-974d-4d840bb5e265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:23.293141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e384-45d9-448b-974d-4d840bb5e265 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:23.293643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e384-45d9-448b-974d-4d840bb5e265 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:23.293752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e384-45d9-448b-974d-4d840bb5e265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:23.320125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:23.418317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857e384-45d9-448b-974d-4d840bb5e265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:23.421181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857e384-45d9-448b-974d-4d840bb5e265 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:23.571560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 496/1040] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:36:23 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 20:36:23.580808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f64f79-93ea-45de-a54c-3cc1398596ad None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:23.683195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:23.763512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f64f79-93ea-45de-a54c-3cc1398596ad None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:23.817623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7792c6-b267-4f18-b38e-c2db2cf840b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:23.818450 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7792c6-b267-4f18-b38e-c2db2cf840b5 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:23.818734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7792c6-b267-4f18-b38e-c2db2cf840b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:23.818995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7792c6-b267-4f18-b38e-c2db2cf840b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:23.832856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7792c6-b267-4f18-b38e-c2db2cf840b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:23.834264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7792c6-b267-4f18-b38e-c2db2cf840b5 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-0c7792c6-b267-4f18-b38e-c2db2cf840b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:23.855513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:23.857141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:23.858878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:23.858878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:23.858878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:23.858878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:23.858878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 545/1041] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:23.868607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857e384-45d9-448b-974d-4d840bb5e265 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:23.928473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:23.999617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857e384-45d9-448b-974d-4d840bb5e265 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:24.059802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab00636-0846-4606-b5c6-a27e449c92cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:24.060939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab00636-0846-4606-b5c6-a27e449c92cc None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:24.061487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab00636-0846-4606-b5c6-a27e449c92cc None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:24.061955 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab00636-0846-4606-b5c6-a27e449c92cc None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:24.063302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab00636-0846-4606-b5c6-a27e449c92cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:24.063834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab00636-0846-4606-b5c6-a27e449c92cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:24.066453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab00636-0846-4606-b5c6-a27e449c92cc None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:24.088812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 497/1042] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:24.097831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7792c6-b267-4f18-b38e-c2db2cf840b5 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:24.275320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7792c6-b267-4f18-b38e-c2db2cf840b5 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:24.294605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:24.336640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134e4d-4d9a-4615-a3ea-39bf5afa436e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:24.338019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134e4d-4d9a-4615-a3ea-39bf5afa436e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:24.338504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134e4d-4d9a-4615-a3ea-39bf5afa436e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:24.339017 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c134e4d-4d9a-4615-a3ea-39bf5afa436e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:24.350837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c134e4d-4d9a-4615-a3ea-39bf5afa436e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:24.353385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c134e4d-4d9a-4615-a3ea-39bf5afa436e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-5c134e4d-4d9a-4615-a3ea-39bf5afa436e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:36:24.379742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:24.380663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:24.381589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:36:24.381589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:36:24.381589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:36:24.381589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:36:24.381589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:36:24.381589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:36:24.381589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:24.381589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:24.386000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 546/1043] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:36:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:24.394799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab00636-0846-4606-b5c6-a27e449c92cc None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:24.563790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab00636-0846-4606-b5c6-a27e449c92cc None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:24.585219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:24.625382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef5d0e-1a90-42a3-8353-669293e72db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:24.626237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef5d0e-1a90-42a3-8353-669293e72db4 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:24.626640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef5d0e-1a90-42a3-8353-669293e72db4 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:24.627027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef5d0e-1a90-42a3-8353-669293e72db4 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:24.628188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef5d0e-1a90-42a3-8353-669293e72db4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:24.628637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef5d0e-1a90-42a3-8353-669293e72db4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:24.631048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef5d0e-1a90-42a3-8353-669293e72db4 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:24.653179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 498/1044] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:36:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:24.660504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134e4d-4d9a-4615-a3ea-39bf5afa436e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:24.839358 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c134e4d-4d9a-4615-a3ea-39bf5afa436e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:24.900182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40713194-8a00-46cb-9d40-2fd769d3997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:24.901255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40713194-8a00-46cb-9d40-2fd769d3997f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:24.901799 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40713194-8a00-46cb-9d40-2fd769d3997f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:24.902282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40713194-8a00-46cb-9d40-2fd769d3997f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:24.903866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40713194-8a00-46cb-9d40-2fd769d3997f None admin] RBAC: Adding query filter params (role.id=7833471323564c3794e65a6b05374aa4, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=e1d09e168fc54f1da06b719989ae8ef0) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:24.904431 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40713194-8a00-46cb-9d40-2fd769d3997f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:24.907080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40713194-8a00-46cb-9d40-2fd769d3997f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:24.917420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 547/1045] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:36:24 2022] GET /identity/v3/role_assignments?user.id=e1d09e168fc54f1da06b719989ae8ef0&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=7833471323564c3794e65a6b05374aa4 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:25.203977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e22302-8bd1-4e1b-b588-014ce88219b2 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:25.204170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e22302-8bd1-4e1b-b588-014ce88219b2 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:25.204455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e22302-8bd1-4e1b-b588-014ce88219b2 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:25.206058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 499/1046] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:25.211486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e36e0c-7822-4d7f-bd33-c686682cf2f4 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:25.211773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e36e0c-7822-4d7f-bd33-c686682cf2f4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:25.212043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e36e0c-7822-4d7f-bd33-c686682cf2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:25.230159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e4e36e0c-7822-4d7f-bd33-c686682cf2f4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:25.231367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e4e36e0c-7822-4d7f-bd33-c686682cf2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:25.268449 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:25.318330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 548/1047] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:25.358660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e22302-8bd1-4e1b-b588-014ce88219b2 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:25.362172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aa059-70b5-4704-9e73-4db5dc0a6de0 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:25.362486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aa059-70b5-4704-9e73-4db5dc0a6de0 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:25.362741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aa059-70b5-4704-9e73-4db5dc0a6de0 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:25.364152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 549/1048] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:25.374693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ab236-2539-4b42-8726-c341a2330d8c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:25.374693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ab236-2539-4b42-8726-c341a2330d8c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:25.374693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ab236-2539-4b42-8726-c341a2330d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:25.397109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-c78ab236-2539-4b42-8726-c341a2330d8c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:25.398489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c78ab236-2539-4b42-8726-c341a2330d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:25.500168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 550/1049] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:36:25.504701 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e22302-8bd1-4e1b-b588-014ce88219b2 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:25.526604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ab236-2539-4b42-8726-c341a2330d8c None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:25.556169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd8b15-58c5-4710-8b31-f293927f6815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:25.557979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd8b15-58c5-4710-8b31-f293927f6815 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:25.558442 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd8b15-58c5-4710-8b31-f293927f6815 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:25.558863 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd8b15-58c5-4710-8b31-f293927f6815 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:25.571414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd8b15-58c5-4710-8b31-f293927f6815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-accd8b15-58c5-4710-8b31-f293927f6815 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:36:25.580340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:25.581557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:36:25.582596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:25.585142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 500/1050] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:36:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:25.592961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd8b15-58c5-4710-8b31-f293927f6815 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:25.643778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:25.667161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:25.678765 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ab236-2539-4b42-8726-c341a2330d8c None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:25.723261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db067e69-0f2c-4aa8-8d93-7665345cb625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:25.724513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db067e69-0f2c-4aa8-8d93-7665345cb625 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:25.724968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db067e69-0f2c-4aa8-8d93-7665345cb625 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:25.725469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db067e69-0f2c-4aa8-8d93-7665345cb625 None admin] PATH_INFO: `/v3/services/image` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:25.726596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db067e69-0f2c-4aa8-8d93-7665345cb625 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:25.728762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db067e69-0f2c-4aa8-8d93-7665345cb625 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-db067e69-0f2c-4aa8-8d93-7665345cb625 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:25.743477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:25.750487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:36:25.752155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:25.753595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd8b15-58c5-4710-8b31-f293927f6815 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:25.753730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 551/1051] 10.222.0.70 () {56 vars in 1105 bytes} [Tue Jul 19 20:36:25 2022] GET /identity/v3/services/image => generated 86 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:25.753730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db067e69-0f2c-4aa8-8d93-7665345cb625 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:25.800272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62e692-ed30-4f5f-b967-da53c73d2940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:25.801867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62e692-ed30-4f5f-b967-da53c73d2940 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:25.802473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62e692-ed30-4f5f-b967-da53c73d2940 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:25.803176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62e692-ed30-4f5f-b967-da53c73d2940 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:25.804678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62e692-ed30-4f5f-b967-da53c73d2940 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:25.805409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62e692-ed30-4f5f-b967-da53c73d2940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:25.807965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c62e692-ed30-4f5f-b967-da53c73d2940 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:25.824208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 501/1052] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:36:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:25.830950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62e692-ed30-4f5f-b967-da53c73d2940 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:25.909520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db067e69-0f2c-4aa8-8d93-7665345cb625 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:25.971247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32091bb9-7a7c-4b14-a216-ebc4a27fb657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:25.971917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32091bb9-7a7c-4b14-a216-ebc4a27fb657 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:25.972193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32091bb9-7a7c-4b14-a216-ebc4a27fb657 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:25.972473 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32091bb9-7a7c-4b14-a216-ebc4a27fb657 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:25.973265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32091bb9-7a7c-4b14-a216-ebc4a27fb657 None admin] RBAC: Adding query filter params (name=image) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:25.973596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32091bb9-7a7c-4b14-a216-ebc4a27fb657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:25.974911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32091bb9-7a7c-4b14-a216-ebc4a27fb657 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:25.988559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 552/1053] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:25.990113 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32091bb9-7a7c-4b14-a216-ebc4a27fb657 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:25.999813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:26.019029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62e692-ed30-4f5f-b967-da53c73d2940 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:26.032785 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:26.080847 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc169a-b274-4f9c-b247-02d2c4158f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:26.082285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc169a-b274-4f9c-b247-02d2c4158f57 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:26.082760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc169a-b274-4f9c-b247-02d2c4158f57 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:26.083251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc169a-b274-4f9c-b247-02d2c4158f57 None admin] PATH_INFO: `/v3/users/9fd68be811e74376b3d94d575a6f0e62` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:26.110741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc169a-b274-4f9c-b247-02d2c4158f57 None admin] RBAC: Authorizing `identity:get_user(user_id=9fd68be811e74376b3d94d575a6f0e62)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:26.113499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc169a-b274-4f9c-b247-02d2c4158f57 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:26.124370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 502/1054] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:36:25 2022] GET /identity/v3/users/9fd68be811e74376b3d94d575a6f0e62 => generated 285 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:26.125786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc169a-b274-4f9c-b247-02d2c4158f57 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:26.151340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32091bb9-7a7c-4b14-a216-ebc4a27fb657 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:26.208790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9a8a5-4e94-4037-ad99-fd69e0f70fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:26.209727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9a8a5-4e94-4037-ad99-fd69e0f70fa5 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:26.210075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9a8a5-4e94-4037-ad99-fd69e0f70fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:26.210959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9a8a5-4e94-4037-ad99-fd69e0f70fa5 None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:26.211278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9a8a5-4e94-4037-ad99-fd69e0f70fa5 None admin] RBAC: Adding query filter params (type=image) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:26.211776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9a8a5-4e94-4037-ad99-fd69e0f70fa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:26.212983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9a8a5-4e94-4037-ad99-fd69e0f70fa5 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:26.220293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 553/1055] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:26.226265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9a8a5-4e94-4037-ad99-fd69e0f70fa5 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:26.284268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc169a-b274-4f9c-b247-02d2c4158f57 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:26.331202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23420e1b-f4ed-4a52-871a-ac17cafeac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:26.332348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23420e1b-f4ed-4a52-871a-ac17cafeac88 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:26.333042 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23420e1b-f4ed-4a52-871a-ac17cafeac88 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:26.333467 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23420e1b-f4ed-4a52-871a-ac17cafeac88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:26.358671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:26.384734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:26.410785 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9a8a5-4e94-4037-ad99-fd69e0f70fa5 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:26.452715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23420e1b-f4ed-4a52-871a-ac17cafeac88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:26.454534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23420e1b-f4ed-4a52-871a-ac17cafeac88 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:26.475550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa9dd1-6561-41cd-91b7-39e0e471d1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:26.476679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa9dd1-6561-41cd-91b7-39e0e471d1d7 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:26.477185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa9dd1-6561-41cd-91b7-39e0e471d1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:26.477736 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa9dd1-6561-41cd-91b7-39e0e471d1d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:26.479701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa9dd1-6561-41cd-91b7-39e0e471d1d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:26.482342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa9dd1-6561-41cd-91b7-39e0e471d1d7 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:26.515561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 554/1056] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jul 19 20:36:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:36:26.606651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 503/1057] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:36:26 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 20:36:26.613293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa9dd1-6561-41cd-91b7-39e0e471d1d7 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:26.755023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa9dd1-6561-41cd-91b7-39e0e471d1d7 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:26.818829 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eef93a-2319-414a-8e1f-12c63aa617de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:26.820350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef93a-2319-414a-8e1f-12c63aa617de None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:26.820850 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef93a-2319-414a-8e1f-12c63aa617de None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:26.821418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef93a-2319-414a-8e1f-12c63aa617de None admin] PATH_INFO: `/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:26.824194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eef93a-2319-414a-8e1f-12c63aa617de None admin] RBAC: Authorizing `identity:get_project(project_id=c0fd7cd02b0645e1a208b86703a1aac8)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:26.827254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eef93a-2319-414a-8e1f-12c63aa617de None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:26.830850 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 555/1058] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:36:26 2022] GET /identity/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8 => generated 311 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:26.838006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23420e1b-f4ed-4a52-871a-ac17cafeac88 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:26.903451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:26.966266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23420e1b-f4ed-4a52-871a-ac17cafeac88 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:27.011533 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee785a-3b6e-4d0c-be09-fec2b28ecd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:27.012206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee785a-3b6e-4d0c-be09-fec2b28ecd8f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:27.012497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee785a-3b6e-4d0c-be09-fec2b28ecd8f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:27.012754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee785a-3b6e-4d0c-be09-fec2b28ecd8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:27.013681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee785a-3b6e-4d0c-be09-fec2b28ecd8f None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=c0fd7cd02b0645e1a208b86703a1aac8, user.id=9fd68be811e74376b3d94d575a6f0e62) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:27.013995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee785a-3b6e-4d0c-be09-fec2b28ecd8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:27.015688 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee785a-3b6e-4d0c-be09-fec2b28ecd8f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:27.024126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 504/1059] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:36:26 2022] GET /identity/v3/role_assignments?user.id=9fd68be811e74376b3d94d575a6f0e62&scope.project.id=c0fd7cd02b0645e1a208b86703a1aac8&role.id=85420c68300642aabfd730fb8fef8347 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:27.394163 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0bac9-6a7c-46d7-a7d5-c076f60866d3 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:27.394574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0bac9-6a7c-46d7-a7d5-c076f60866d3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:27.394844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0bac9-6a7c-46d7-a7d5-c076f60866d3 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:27.400320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 556/1060] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:27.403468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8942cb-79e8-40d5-8409-0ce19d5c1bbf None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:27.403805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8942cb-79e8-40d5-8409-0ce19d5c1bbf None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:27.404312 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8942cb-79e8-40d5-8409-0ce19d5c1bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:27.431134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ff8942cb-79e8-40d5-8409-0ce19d5c1bbf None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:27.437925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ff8942cb-79e8-40d5-8409-0ce19d5c1bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:27.601420 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 505/1061] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:27.654683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0bac9-6a7c-46d7-a7d5-c076f60866d3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:27.671305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:27.864680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0bac9-6a7c-46d7-a7d5-c076f60866d3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:27.944290 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef74b2-7477-48e5-872d-318f82d00056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:27.945127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef74b2-7477-48e5-872d-318f82d00056 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:27.945590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef74b2-7477-48e5-872d-318f82d00056 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:27.945919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef74b2-7477-48e5-872d-318f82d00056 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:27.970116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef74b2-7477-48e5-872d-318f82d00056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-36ef74b2-7477-48e5-872d-318f82d00056 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:36:27.987353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:27.989191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:27.997117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 557/1062] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jul 19 20:36:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:28.005348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8942cb-79e8-40d5-8409-0ce19d5c1bbf None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:28.101969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:28.214912 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8942cb-79e8-40d5-8409-0ce19d5c1bbf None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:28.283696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f93686-fc80-4725-b2db-02b651b58966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:28.284903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f93686-fc80-4725-b2db-02b651b58966 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:28.285429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f93686-fc80-4725-b2db-02b651b58966 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:28.285938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f93686-fc80-4725-b2db-02b651b58966 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:28.287385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f93686-fc80-4725-b2db-02b651b58966 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:28.287943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f93686-fc80-4725-b2db-02b651b58966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:28.290690 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f93686-fc80-4725-b2db-02b651b58966 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:28.309633 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 506/1063] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jul 19 20:36:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:28.319226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef74b2-7477-48e5-872d-318f82d00056 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:28.451467 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:28.503174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef74b2-7477-48e5-872d-318f82d00056 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:28.554530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f55e26-d072-47e2-a29a-2ae6dd060916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:28.555875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f55e26-d072-47e2-a29a-2ae6dd060916 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:28.556381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f55e26-d072-47e2-a29a-2ae6dd060916 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:28.557438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f55e26-d072-47e2-a29a-2ae6dd060916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:28.570761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f55e26-d072-47e2-a29a-2ae6dd060916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:28.572245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f55e26-d072-47e2-a29a-2ae6dd060916 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-59f55e26-d072-47e2-a29a-2ae6dd060916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:28.597894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:28.600183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:28.601411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:28.601411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:28.601411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:28.601411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:28.601411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 558/1064] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:28.608963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f93686-fc80-4725-b2db-02b651b58966 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:28.745760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f93686-fc80-4725-b2db-02b651b58966 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:28.754468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:28.785106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590db4d-228e-44ac-beb4-6af05dda6b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:28.785794 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590db4d-228e-44ac-beb4-6af05dda6b3c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:28.786074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590db4d-228e-44ac-beb4-6af05dda6b3c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:28.786343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590db4d-228e-44ac-beb4-6af05dda6b3c None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:28.787073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590db4d-228e-44ac-beb4-6af05dda6b3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:28.787367 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590db4d-228e-44ac-beb4-6af05dda6b3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:28.788729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590db4d-228e-44ac-beb4-6af05dda6b3c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:28.806672 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 507/1065] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:28.808508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f55e26-d072-47e2-a29a-2ae6dd060916 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:28.995500 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f55e26-d072-47e2-a29a-2ae6dd060916 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:29.030672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:29.068099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce914774-ad17-4894-90b3-70a6c6a27190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:29.069414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce914774-ad17-4894-90b3-70a6c6a27190 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:29.069887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce914774-ad17-4894-90b3-70a6c6a27190 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:29.070349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce914774-ad17-4894-90b3-70a6c6a27190 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:29.092475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce914774-ad17-4894-90b3-70a6c6a27190 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:29.095139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce914774-ad17-4894-90b3-70a6c6a27190 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-ce914774-ad17-4894-90b3-70a6c6a27190 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:36:29.121603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:29.123864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:29.125897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:29.125897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:29.125897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:36:29.125897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:36:29.125897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:36:29.125897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:36:29.125897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:36:29.125897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:36:29.125897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:36:29.125897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:29.125897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 559/1066] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jul 19 20:36:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:29.139412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590db4d-228e-44ac-beb4-6af05dda6b3c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:29.266432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590db4d-228e-44ac-beb4-6af05dda6b3c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:29.283234 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1b02e-216a-428b-89b0-6b17595fce3b None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:29.283234 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1b02e-216a-428b-89b0-6b17595fce3b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:29.283493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1b02e-216a-428b-89b0-6b17595fce3b None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:29.284851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 560/1067] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:36:29.289956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094eb8f1-e9ee-4ff6-ba8a-862933d7b364 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:29.290318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094eb8f1-e9ee-4ff6-ba8a-862933d7b364 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:29.290503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094eb8f1-e9ee-4ff6-ba8a-862933d7b364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:29.306244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785901e-f4bb-45de-8f68-84ab560912dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:29.307497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785901e-f4bb-45de-8f68-84ab560912dd None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:29.307782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785901e-f4bb-45de-8f68-84ab560912dd None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:29.308067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785901e-f4bb-45de-8f68-84ab560912dd None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:29.309063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785901e-f4bb-45de-8f68-84ab560912dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:29.309247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785901e-f4bb-45de-8f68-84ab560912dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:29.310013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-094eb8f1-e9ee-4ff6-ba8a-862933d7b364 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:29.310697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785901e-f4bb-45de-8f68-84ab560912dd None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:29.311226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-094eb8f1-e9ee-4ff6-ba8a-862933d7b364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:29.337726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 508/1068] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jul 19 20:36:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:29.339359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785901e-f4bb-45de-8f68-84ab560912dd None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:29.379683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:29.405261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 561/1069] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:36:29.406675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3921591-7c2d-41ec-9e3d-e34663c37dd8 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:29.406963 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3921591-7c2d-41ec-9e3d-e34663c37dd8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:29.407243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3921591-7c2d-41ec-9e3d-e34663c37dd8 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:29.408598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 562/1070] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:29.414310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcdd02-7aeb-46d6-b2a6-204273b0c358 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:29.414585 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcdd02-7aeb-46d6-b2a6-204273b0c358 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:29.414864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcdd02-7aeb-46d6-b2a6-204273b0c358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:29.434358 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-17fcdd02-7aeb-46d6-b2a6-204273b0c358 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:29.435606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-17fcdd02-7aeb-46d6-b2a6-204273b0c358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:29.474615 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785901e-f4bb-45de-8f68-84ab560912dd None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:29.514499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9274e58-a132-49d3-a0de-3840253fe2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:29.515301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9274e58-a132-49d3-a0de-3840253fe2ed None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:29.515581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9274e58-a132-49d3-a0de-3840253fe2ed None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:29.515857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9274e58-a132-49d3-a0de-3840253fe2ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:29.519354 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 563/1071] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:36:29.521392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fcdd02-7aeb-46d6-b2a6-204273b0c358 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:29.530116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9274e58-a132-49d3-a0de-3840253fe2ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:29.531757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9274e58-a132-49d3-a0de-3840253fe2ed None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-f9274e58-a132-49d3-a0de-3840253fe2ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:29.552503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:29.553610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:29.554707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:29.554707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:29.554707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:29.554707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:29.554707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 509/1072] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:29.555073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9274e58-a132-49d3-a0de-3840253fe2ed None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:29.581561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:29.636703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:29.643878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fcdd02-7aeb-46d6-b2a6-204273b0c358 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:29.660746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9274e58-a132-49d3-a0de-3840253fe2ed None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:29.679324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e4a7b-04dc-47c8-9145-a330d6fab474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:29.680181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e4a7b-04dc-47c8-9145-a330d6fab474 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:29.680463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e4a7b-04dc-47c8-9145-a330d6fab474 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:29.680742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e4a7b-04dc-47c8-9145-a330d6fab474 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:29.691150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e4a7b-04dc-47c8-9145-a330d6fab474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:36:29.696501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f6d48-21f5-4537-8364-196f54951fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:29.697156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f6d48-21f5-4537-8364-196f54951fa6 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:29.697503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f6d48-21f5-4537-8364-196f54951fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:29.697783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f6d48-21f5-4537-8364-196f54951fa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-839e4a7b-04dc-47c8-9145-a330d6fab474 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:36:29.698855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:29.700226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 564/1073] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:36:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:29.702631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e4a7b-04dc-47c8-9145-a330d6fab474 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:29.772674 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f6d48-21f5-4537-8364-196f54951fa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:29.774260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f6d48-21f5-4537-8364-196f54951fa6 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:29.804419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:29.809024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e4a7b-04dc-47c8-9145-a330d6fab474 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:29.846487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1372b-34ae-4a57-b338-342dd5b570b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:29.847148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1372b-34ae-4a57-b338-342dd5b570b2 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:29.847436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1372b-34ae-4a57-b338-342dd5b570b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:29.847715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1372b-34ae-4a57-b338-342dd5b570b2 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:29.848448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1372b-34ae-4a57-b338-342dd5b570b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:29.848759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1372b-34ae-4a57-b338-342dd5b570b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:29.850263 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1372b-34ae-4a57-b338-342dd5b570b2 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:29.852679 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:29.863167 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 565/1074] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:29.864007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1372b-34ae-4a57-b338-342dd5b570b2 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:29.882479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 510/1075] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:36:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:36:29.883339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f6d48-21f5-4537-8364-196f54951fa6 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:29.970969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1372b-34ae-4a57-b338-342dd5b570b2 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:29.984847 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f6d48-21f5-4537-8364-196f54951fa6 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:30.010968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46b6ff-c488-4345-9460-ea76546c08f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:30.011649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46b6ff-c488-4345-9460-ea76546c08f0 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:30.011932 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46b6ff-c488-4345-9460-ea76546c08f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:30.012210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46b6ff-c488-4345-9460-ea76546c08f0 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:30.012963 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46b6ff-c488-4345-9460-ea76546c08f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:30.013268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46b6ff-c488-4345-9460-ea76546c08f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:30.014664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46b6ff-c488-4345-9460-ea76546c08f0 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:30.020684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542c504-4eb7-4fb5-ad30-3e762f670400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:30.021509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542c504-4eb7-4fb5-ad30-3e762f670400 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:30.021790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542c504-4eb7-4fb5-ad30-3e762f670400 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:30.022059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542c504-4eb7-4fb5-ad30-3e762f670400 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:30.023146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 566/1076] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:36:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:30.023971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46b6ff-c488-4345-9460-ea76546c08f0 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:30.033881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542c504-4eb7-4fb5-ad30-3e762f670400 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:30.035352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542c504-4eb7-4fb5-ad30-3e762f670400 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:30.047086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-0542c504-4eb7-4fb5-ad30-3e762f670400 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:36:30.051032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:30.052647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:30.053934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:36:30.053934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:36:30.053934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:36:30.053934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:36:30.053934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:36:30.053934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:36:30.053934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:30.053934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:30.053934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 511/1077] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:36:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:30.053934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542c504-4eb7-4fb5-ad30-3e762f670400 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:30.090320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:30.138937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46b6ff-c488-4345-9460-ea76546c08f0 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:30.158184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542c504-4eb7-4fb5-ad30-3e762f670400 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:30.175156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73039f37-d4f2-476b-ac19-287264223eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:30.175957 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73039f37-d4f2-476b-ac19-287264223eca None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:30.176242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73039f37-d4f2-476b-ac19-287264223eca None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:30.176521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73039f37-d4f2-476b-ac19-287264223eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:30.187283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73039f37-d4f2-476b-ac19-287264223eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:30.188723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73039f37-d4f2-476b-ac19-287264223eca None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:30.191919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dff06a-8d5c-41b5-86f2-d44d89897dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:30.192785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dff06a-8d5c-41b5-86f2-d44d89897dd1 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:30.193064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dff06a-8d5c-41b5-86f2-d44d89897dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:30.193334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dff06a-8d5c-41b5-86f2-d44d89897dd1 None admin] PATH_INFO: `/v3/users/9fd68be811e74376b3d94d575a6f0e62` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:30.194543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dff06a-8d5c-41b5-86f2-d44d89897dd1 None admin] RBAC: Authorizing `identity:get_user(user_id=9fd68be811e74376b3d94d575a6f0e62)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:30.195959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dff06a-8d5c-41b5-86f2-d44d89897dd1 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:30.203072 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 512/1078] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:36:30 2022] GET /identity/v3/users/9fd68be811e74376b3d94d575a6f0e62 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:30.203072 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dff06a-8d5c-41b5-86f2-d44d89897dd1 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-73039f37-d4f2-476b-ac19-287264223eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:30.206274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:30.207152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:30.208039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:30.208039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:30.208039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:30.208039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:30.208039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 567/1079] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:30.208584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73039f37-d4f2-476b-ac19-287264223eca None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:30.279878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:30.301317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dff06a-8d5c-41b5-86f2-d44d89897dd1 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:30.303627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:30.311110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73039f37-d4f2-476b-ac19-287264223eca None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:30.337087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5194c-758c-4d3f-99d7-ea58d3b91935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:30.337778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5194c-758c-4d3f-99d7-ea58d3b91935 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:30.338094 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5194c-758c-4d3f-99d7-ea58d3b91935 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:30.338371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5194c-758c-4d3f-99d7-ea58d3b91935 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:30.339151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5194c-758c-4d3f-99d7-ea58d3b91935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:30.339462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5194c-758c-4d3f-99d7-ea58d3b91935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:30.340923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5194c-758c-4d3f-99d7-ea58d3b91935 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:30.344701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17f38a-6504-4d89-85fb-ad4c70833879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:30.345611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17f38a-6504-4d89-85fb-ad4c70833879 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:30.345899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17f38a-6504-4d89-85fb-ad4c70833879 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:30.346177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17f38a-6504-4d89-85fb-ad4c70833879 None admin] PATH_INFO: `/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:30.347444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17f38a-6504-4d89-85fb-ad4c70833879 None admin] RBAC: Authorizing `identity:get_project(project_id=c0fd7cd02b0645e1a208b86703a1aac8)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:30.348850 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17f38a-6504-4d89-85fb-ad4c70833879 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:30.350656 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 568/1080] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:36:30 2022] GET /identity/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:30.351343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17f38a-6504-4d89-85fb-ad4c70833879 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:30.355891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 513/1081] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:36:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:30.361335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5194c-758c-4d3f-99d7-ea58d3b91935 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:30.457100 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17f38a-6504-4d89-85fb-ad4c70833879 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:30.463691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5194c-758c-4d3f-99d7-ea58d3b91935 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:30.496673 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88b3bd-8e62-4ae7-9c44-86e6a9a6cb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:30.497336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88b3bd-8e62-4ae7-9c44-86e6a9a6cb61 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:30.497656 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88b3bd-8e62-4ae7-9c44-86e6a9a6cb61 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:30.497930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88b3bd-8e62-4ae7-9c44-86e6a9a6cb61 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:30.498667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88b3bd-8e62-4ae7-9c44-86e6a9a6cb61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:30.498967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88b3bd-8e62-4ae7-9c44-86e6a9a6cb61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:30.500139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918eb4b5-49da-49b2-9d7b-42765c24ade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:30.500339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88b3bd-8e62-4ae7-9c44-86e6a9a6cb61 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:30.501034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918eb4b5-49da-49b2-9d7b-42765c24ade7 None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:30.501313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918eb4b5-49da-49b2-9d7b-42765c24ade7 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:30.502056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918eb4b5-49da-49b2-9d7b-42765c24ade7 None admin] PATH_INFO: `/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8/users/9fd68be811e74376b3d94d575a6f0e62/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:30.503773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918eb4b5-49da-49b2-9d7b-42765c24ade7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fd7cd02b0645e1a208b86703a1aac8, user_id=9fd68be811e74376b3d94d575a6f0e62, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:30.505219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918eb4b5-49da-49b2-9d7b-42765c24ade7 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:30.513541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:30.517750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 569/1082] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:30.518936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88b3bd-8e62-4ae7-9c44-86e6a9a6cb61 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:30.521977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 514/1083] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:36:30 2022] PUT /identity/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8/users/9fd68be811e74376b3d94d575a6f0e62/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:36:30.654517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88b3bd-8e62-4ae7-9c44-86e6a9a6cb61 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:30.698027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174b2d6-d57d-4de5-a42e-79909171573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:30.699158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174b2d6-d57d-4de5-a42e-79909171573c None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:30.699359 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174b2d6-d57d-4de5-a42e-79909171573c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:30.699928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174b2d6-d57d-4de5-a42e-79909171573c None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/e1d09e168fc54f1da06b719989ae8ef0/roles/7833471323564c3794e65a6b05374aa4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:30.702646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174b2d6-d57d-4de5-a42e-79909171573c None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=e1d09e168fc54f1da06b719989ae8ef0, role_id=7833471323564c3794e65a6b05374aa4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:30.704399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174b2d6-d57d-4de5-a42e-79909171573c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:30.718930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 570/1084] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:36:30 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/e1d09e168fc54f1da06b719989ae8ef0/roles/7833471323564c3794e65a6b05374aa4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:36:32.611919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749206-1322-4917-8c30-005826150c02 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:32.612499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749206-1322-4917-8c30-005826150c02 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:32.612499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749206-1322-4917-8c30-005826150c02 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:32.613898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 515/1085] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:32.619323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18ca9-86db-4853-84b4-bb5914555e74 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:32.619779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18ca9-86db-4853-84b4-bb5914555e74 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:32.619871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18ca9-86db-4853-84b4-bb5914555e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:32.636627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-94b18ca9-86db-4853-84b4-bb5914555e74 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:32.637788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-94b18ca9-86db-4853-84b4-bb5914555e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:32.698345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:32.710846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 571/1086] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:32.734646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02749206-1322-4917-8c30-005826150c02 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:32.761871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:32.779193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17926ac-ce22-41ee-a401-62c87d949f43 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:32.779485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17926ac-ce22-41ee-a401-62c87d949f43 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:32.779759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17926ac-ce22-41ee-a401-62c87d949f43 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:32.781122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 572/1087] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:36:32.786267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2270e96-4d8e-4120-b35b-44e378a6d4b5 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:32.786633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2270e96-4d8e-4120-b35b-44e378a6d4b5 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:32.786908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2270e96-4d8e-4120-b35b-44e378a6d4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:32.803234 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a2270e96-4d8e-4120-b35b-44e378a6d4b5 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:32.804414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a2270e96-4d8e-4120-b35b-44e378a6d4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:32.841276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02749206-1322-4917-8c30-005826150c02 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:32.871233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 573/1088] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:36:32.874480 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a353bc-2895-41da-80c3-b6cd7054795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:32.875358 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a353bc-2895-41da-80c3-b6cd7054795b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:32.875771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a353bc-2895-41da-80c3-b6cd7054795b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:32.876019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a353bc-2895-41da-80c3-b6cd7054795b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:32.888672 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a353bc-2895-41da-80c3-b6cd7054795b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:36:32.896478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2270e96-4d8e-4120-b35b-44e378a6d4b5 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-82a353bc-2895-41da-80c3-b6cd7054795b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:36:32.896647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:32.898024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:32.898859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 516/1089] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:36:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:32.905819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a353bc-2895-41da-80c3-b6cd7054795b None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:32.997318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:33.003938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2270e96-4d8e-4120-b35b-44e378a6d4b5 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:33.014117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a353bc-2895-41da-80c3-b6cd7054795b None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:33.025255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:33.040873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2858d54-2a9b-449a-9f04-6d473e1f741d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:33.041722 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2858d54-2a9b-449a-9f04-6d473e1f741d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:33.042005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2858d54-2a9b-449a-9f04-6d473e1f741d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:33.042346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2858d54-2a9b-449a-9f04-6d473e1f741d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:33.049814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4656c24-d92f-437a-afe3-937fe7587ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:33.050489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4656c24-d92f-437a-afe3-937fe7587ebb None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:33.050771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4656c24-d92f-437a-afe3-937fe7587ebb None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:33.051049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4656c24-d92f-437a-afe3-937fe7587ebb None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:33.051806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4656c24-d92f-437a-afe3-937fe7587ebb None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:33.052118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4656c24-d92f-437a-afe3-937fe7587ebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:33.052822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2858d54-2a9b-449a-9f04-6d473e1f741d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:36:33.053504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4656c24-d92f-437a-afe3-937fe7587ebb None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-d2858d54-2a9b-449a-9f04-6d473e1f741d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:36:33.061082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:33.062293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:36:33.063296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:33.064067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 517/1090] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:36:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:33.064928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 574/1091] 10.222.0.70 () {56 vars in 1120 bytes} [Tue Jul 19 20:36:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:33.067377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4656c24-d92f-437a-afe3-937fe7587ebb None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:33.071327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2858d54-2a9b-449a-9f04-6d473e1f741d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:33.173396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4656c24-d92f-437a-afe3-937fe7587ebb None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:33.178594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2858d54-2a9b-449a-9f04-6d473e1f741d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:33.211983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef03e6e-58c8-4ad2-8982-5c282a0c1556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:33.213092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef03e6e-58c8-4ad2-8982-5c282a0c1556 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:33.213496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef03e6e-58c8-4ad2-8982-5c282a0c1556 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:33.213838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef03e6e-58c8-4ad2-8982-5c282a0c1556 None admin] PATH_INFO: `/v3/users/9fd68be811e74376b3d94d575a6f0e62` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:33.213971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dab87a-e25d-4b14-8c61-27f5f1762ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:33.214661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dab87a-e25d-4b14-8c61-27f5f1762ab4 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:33.214941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dab87a-e25d-4b14-8c61-27f5f1762ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:33.215212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dab87a-e25d-4b14-8c61-27f5f1762ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:33.215335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef03e6e-58c8-4ad2-8982-5c282a0c1556 None admin] RBAC: Authorizing `identity:get_user(user_id=9fd68be811e74376b3d94d575a6f0e62)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:33.216078 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dab87a-e25d-4b14-8c61-27f5f1762ab4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:33.216381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dab87a-e25d-4b14-8c61-27f5f1762ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:33.217148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef03e6e-58c8-4ad2-8982-5c282a0c1556 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:33.217741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dab87a-e25d-4b14-8c61-27f5f1762ab4 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:33.219044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 518/1092] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:36:33 2022] GET /identity/v3/users/9fd68be811e74376b3d94d575a6f0e62 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:33.227912 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef03e6e-58c8-4ad2-8982-5c282a0c1556 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:33.235518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 575/1093] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jul 19 20:36:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:33.236549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dab87a-e25d-4b14-8c61-27f5f1762ab4 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:33.246779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:33.301166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:33.336791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef03e6e-58c8-4ad2-8982-5c282a0c1556 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:33.358715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dab87a-e25d-4b14-8c61-27f5f1762ab4 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:33.370258 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ef260-5635-4afa-8cef-cd130de437d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:33.370908 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ef260-5635-4afa-8cef-cd130de437d8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:33.371194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ef260-5635-4afa-8cef-cd130de437d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:33.371473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ef260-5635-4afa-8cef-cd130de437d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:33.397821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130662d-f9ec-499d-9c1b-1b7b5a8df883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:33.398617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130662d-f9ec-499d-9c1b-1b7b5a8df883 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:33.398907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130662d-f9ec-499d-9c1b-1b7b5a8df883 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:33.399190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130662d-f9ec-499d-9c1b-1b7b5a8df883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:33.410058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130662d-f9ec-499d-9c1b-1b7b5a8df883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:33.411503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130662d-f9ec-499d-9c1b-1b7b5a8df883 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-8130662d-f9ec-499d-9c1b-1b7b5a8df883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:33.428698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:33.429713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:33.430752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:33.430752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:33.430752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:33.430752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:33.430752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 576/1094] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:33.436014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130662d-f9ec-499d-9c1b-1b7b5a8df883 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:33.438598 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ef260-5635-4afa-8cef-cd130de437d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:33.440032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ef260-5635-4afa-8cef-cd130de437d8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:33.455900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:33.529975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:33.537748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130662d-f9ec-499d-9c1b-1b7b5a8df883 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:33.538721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 519/1095] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:36:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 20:36:33.544750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ef260-5635-4afa-8cef-cd130de437d8 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:33.572532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bef7e-8eb9-42b0-8324-3b8127310eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:33.573208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bef7e-8eb9-42b0-8324-3b8127310eb4 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:33.573534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bef7e-8eb9-42b0-8324-3b8127310eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:33.573814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bef7e-8eb9-42b0-8324-3b8127310eb4 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:33.574545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bef7e-8eb9-42b0-8324-3b8127310eb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:33.575113 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bef7e-8eb9-42b0-8324-3b8127310eb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:33.576478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bef7e-8eb9-42b0-8324-3b8127310eb4 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:33.588423 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 577/1096] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:33.594158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bef7e-8eb9-42b0-8324-3b8127310eb4 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:33.645782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ef260-5635-4afa-8cef-cd130de437d8 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:33.663771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:33.685070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d134204-f532-4b78-86e9-c0edb5bbdc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:33.685852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d134204-f532-4b78-86e9-c0edb5bbdc57 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:33.686125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d134204-f532-4b78-86e9-c0edb5bbdc57 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:33.686407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d134204-f532-4b78-86e9-c0edb5bbdc57 None admin] PATH_INFO: `/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:33.687630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d134204-f532-4b78-86e9-c0edb5bbdc57 None admin] RBAC: Authorizing `identity:get_project(project_id=c0fd7cd02b0645e1a208b86703a1aac8)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:33.689074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d134204-f532-4b78-86e9-c0edb5bbdc57 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:33.690826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 520/1097] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:36:33 2022] GET /identity/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:33.696764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d134204-f532-4b78-86e9-c0edb5bbdc57 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:33.701928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bef7e-8eb9-42b0-8324-3b8127310eb4 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:33.736718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff7138-1efb-4feb-9e2b-1996cf04b18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:33.737546 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff7138-1efb-4feb-9e2b-1996cf04b18a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:33.737834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff7138-1efb-4feb-9e2b-1996cf04b18a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:33.738110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff7138-1efb-4feb-9e2b-1996cf04b18a None admin] PATH_INFO: `/v3/users/nova` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:33.746073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff7138-1efb-4feb-9e2b-1996cf04b18a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:33.747515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff7138-1efb-4feb-9e2b-1996cf04b18a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-4fff7138-1efb-4feb-9e2b-1996cf04b18a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:36:33.762058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:33.763072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:33.764016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:33.764016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:33.764016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:36:33.764016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:36:33.764016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:36:33.764016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:36:33.764016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:36:33.764016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:36:33.764016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:36:33.764016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:33.764016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 578/1098] 10.222.0.70 () {56 vars in 1093 bytes} [Tue Jul 19 20:36:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:36:33.768989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff7138-1efb-4feb-9e2b-1996cf04b18a None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:33.784592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:33.796912 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d134204-f532-4b78-86e9-c0edb5bbdc57 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:33.832204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09440d47-2ea4-4adf-9dd1-be18744db74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:33.832854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09440d47-2ea4-4adf-9dd1-be18744db74d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:33.833139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09440d47-2ea4-4adf-9dd1-be18744db74d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:33.833436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09440d47-2ea4-4adf-9dd1-be18744db74d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:33.834257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09440d47-2ea4-4adf-9dd1-be18744db74d None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=c0fd7cd02b0645e1a208b86703a1aac8, user.id=9fd68be811e74376b3d94d575a6f0e62) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:33.834564 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09440d47-2ea4-4adf-9dd1-be18744db74d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:33.835972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09440d47-2ea4-4adf-9dd1-be18744db74d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:33.841717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 521/1099] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:36:33 2022] GET /identity/v3/role_assignments?user.id=9fd68be811e74376b3d94d575a6f0e62&scope.project.id=c0fd7cd02b0645e1a208b86703a1aac8&role.id=85420c68300642aabfd730fb8fef8347 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:33.874422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff7138-1efb-4feb-9e2b-1996cf04b18a None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:33.912082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdd938-8a06-4e65-874c-9e102cabb41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:33.912788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdd938-8a06-4e65-874c-9e102cabb41b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:33.913073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdd938-8a06-4e65-874c-9e102cabb41b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:33.913352 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdd938-8a06-4e65-874c-9e102cabb41b None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:33.914266 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdd938-8a06-4e65-874c-9e102cabb41b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:33.914581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdd938-8a06-4e65-874c-9e102cabb41b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:33.916123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdd938-8a06-4e65-874c-9e102cabb41b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:33.937574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 579/1100] 10.222.0.70 () {56 vars in 1133 bytes} [Tue Jul 19 20:36:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:33.943931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09440d47-2ea4-4adf-9dd1-be18744db74d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:33.999080 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:34.060055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09440d47-2ea4-4adf-9dd1-be18744db74d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:34.094289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa0130-df04-4b02-91ae-771aa521b0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:34.094838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa0130-df04-4b02-91ae-771aa521b0f9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:34.095119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa0130-df04-4b02-91ae-771aa521b0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:34.095391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa0130-df04-4b02-91ae-771aa521b0f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:34.164414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa0130-df04-4b02-91ae-771aa521b0f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:34.165898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa0130-df04-4b02-91ae-771aa521b0f9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:34.211457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:34.267540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 522/1101] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:36:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 20:36:34.273386 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdd938-8a06-4e65-874c-9e102cabb41b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:34.345382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:34.379665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdd938-8a06-4e65-874c-9e102cabb41b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:34.414616 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de73f80-8131-42b5-bebd-4e80ccd61f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:34.415402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de73f80-8131-42b5-bebd-4e80ccd61f62 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:34.415677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de73f80-8131-42b5-bebd-4e80ccd61f62 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:34.415950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de73f80-8131-42b5-bebd-4e80ccd61f62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:34.427713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de73f80-8131-42b5-bebd-4e80ccd61f62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:34.429159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de73f80-8131-42b5-bebd-4e80ccd61f62 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-3de73f80-8131-42b5-bebd-4e80ccd61f62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:36:34.447556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:34.448519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:34.449462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:36:34.449462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:36:34.449462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:36:34.449462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:34.449462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 580/1102] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:34.454891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa0130-df04-4b02-91ae-771aa521b0f9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:34.566217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa0130-df04-4b02-91ae-771aa521b0f9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:34.605110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b7665-0582-4772-935e-3b470cd791a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:34.605829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b7665-0582-4772-935e-3b470cd791a6 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:34.606116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b7665-0582-4772-935e-3b470cd791a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:34.606390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b7665-0582-4772-935e-3b470cd791a6 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:34.607170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b7665-0582-4772-935e-3b470cd791a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:34.607480 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b7665-0582-4772-935e-3b470cd791a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:34.608924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b7665-0582-4772-935e-3b470cd791a6 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:34.622101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 523/1103] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:34.628147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de73f80-8131-42b5-bebd-4e80ccd61f62 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:34.737374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de73f80-8131-42b5-bebd-4e80ccd61f62 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:34.764213 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:34.777833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dca29f-2d69-4ad7-9b1a-541d18967ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:34.778732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dca29f-2d69-4ad7-9b1a-541d18967ca5 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:34.779026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dca29f-2d69-4ad7-9b1a-541d18967ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:34.779310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dca29f-2d69-4ad7-9b1a-541d18967ca5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:34.787178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dca29f-2d69-4ad7-9b1a-541d18967ca5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:34.788625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dca29f-2d69-4ad7-9b1a-541d18967ca5 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-25dca29f-2d69-4ad7-9b1a-541d18967ca5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:36:34.803056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:34.803999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:34.804881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:36:34.804881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:36:34.804881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:36:34.804881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:36:34.804881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:36:34.804881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:36:34.804881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:36:34.804881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:36:34.804881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 581/1104] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:36:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:34.810668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b7665-0582-4772-935e-3b470cd791a6 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:34.820524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:34.924271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b7665-0582-4772-935e-3b470cd791a6 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:34.960408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1428c-c33a-48a0-ac66-b031df8beafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:34.961075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1428c-c33a-48a0-ac66-b031df8beafb None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:34.961369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1428c-c33a-48a0-ac66-b031df8beafb None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:34.961649 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1428c-c33a-48a0-ac66-b031df8beafb None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:34.962445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1428c-c33a-48a0-ac66-b031df8beafb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:34.962745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1428c-c33a-48a0-ac66-b031df8beafb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:34.964261 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1428c-c33a-48a0-ac66-b031df8beafb None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:34.977135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 524/1105] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:36:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:34.983847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dca29f-2d69-4ad7-9b1a-541d18967ca5 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:35.093165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dca29f-2d69-4ad7-9b1a-541d18967ca5 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:35.135930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ef7b0-0539-472e-81f7-93929ed44a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:35.136597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ef7b0-0539-472e-81f7-93929ed44a6f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:35.136884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ef7b0-0539-472e-81f7-93929ed44a6f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:35.137165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ef7b0-0539-472e-81f7-93929ed44a6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:35.138050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ef7b0-0539-472e-81f7-93929ed44a6f None admin] RBAC: Adding query filter params (role.id=7833471323564c3794e65a6b05374aa4, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=e1d09e168fc54f1da06b719989ae8ef0) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:35.138368 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ef7b0-0539-472e-81f7-93929ed44a6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:35.139812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ef7b0-0539-472e-81f7-93929ed44a6f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:35.146341 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 582/1106] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:36:34 2022] GET /identity/v3/role_assignments?user.id=e1d09e168fc54f1da06b719989ae8ef0&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=7833471323564c3794e65a6b05374aa4 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:36.008680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2edd76-ae32-4147-a55d-b117519721a5 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:36.009218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2edd76-ae32-4147-a55d-b117519721a5 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:36.009663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2edd76-ae32-4147-a55d-b117519721a5 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:36.011165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 525/1107] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:36.017081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a413e4-ae8c-4f47-bbe1-d9384bb0670e None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:36.017623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a413e4-ae8c-4f47-bbe1-d9384bb0670e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:36.017726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a413e4-ae8c-4f47-bbe1-d9384bb0670e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:36.035600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-17a413e4-ae8c-4f47-bbe1-d9384bb0670e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:36.036825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-17a413e4-ae8c-4f47-bbe1-d9384bb0670e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:36.092212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:36.118308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 583/1108] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:36.144314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2edd76-ae32-4147-a55d-b117519721a5 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:36.218870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:36.255770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2edd76-ae32-4147-a55d-b117519721a5 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:36.290880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3af122-182e-4020-867a-b158a989fcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:36.294268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3af122-182e-4020-867a-b158a989fcdd None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:36.294551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3af122-182e-4020-867a-b158a989fcdd None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:36.294826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3af122-182e-4020-867a-b158a989fcdd None admin] PATH_INFO: `/v3/domains/default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:36.296130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3af122-182e-4020-867a-b158a989fcdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:36.297607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3af122-182e-4020-867a-b158a989fcdd None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:36.299264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 526/1109] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:36.305695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a413e4-ae8c-4f47-bbe1-d9384bb0670e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:36.407904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a413e4-ae8c-4f47-bbe1-d9384bb0670e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:36.443086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5a9f1-e726-423f-b80a-efc67da6656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:36.443751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5a9f1-e726-423f-b80a-efc67da6656d None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:36.444028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5a9f1-e726-423f-b80a-efc67da6656d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:36.444297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5a9f1-e726-423f-b80a-efc67da6656d None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:36.444993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5a9f1-e726-423f-b80a-efc67da6656d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:36.446448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5a9f1-e726-423f-b80a-efc67da6656d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:36.467921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 584/1110] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:36:36 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:36:38.594356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189f460-8435-436b-aca9-bd4cb5978f86 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:38.595185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189f460-8435-436b-aca9-bd4cb5978f86 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:38.595376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189f460-8435-436b-aca9-bd4cb5978f86 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:38.596771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 527/1111] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:36:38.602784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc797b71-bceb-4084-a9c2-6ab1923ff00c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:38.603292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc797b71-bceb-4084-a9c2-6ab1923ff00c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:38.603418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc797b71-bceb-4084-a9c2-6ab1923ff00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:38.622647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-bc797b71-bceb-4084-a9c2-6ab1923ff00c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:38.623933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-bc797b71-bceb-4084-a9c2-6ab1923ff00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:38.653902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:38.699578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 585/1112] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:36:38.723543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189f460-8435-436b-aca9-bd4cb5978f86 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:38.836165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189f460-8435-436b-aca9-bd4cb5978f86 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:38.871437 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08bca6-5b97-41bc-9560-76a5706dd5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:38.872326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08bca6-5b97-41bc-9560-76a5706dd5d3 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:38.872653 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08bca6-5b97-41bc-9560-76a5706dd5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:38.872892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08bca6-5b97-41bc-9560-76a5706dd5d3 None admin] PATH_INFO: `/v3/roles/0044294fcceb4b6ba0cb27567f2761d4` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:38.874578 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08bca6-5b97-41bc-9560-76a5706dd5d3 None admin] RBAC: Authorizing `identity:get_role(role_id=0044294fcceb4b6ba0cb27567f2761d4)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:38.875965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08bca6-5b97-41bc-9560-76a5706dd5d3 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:38.877333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 528/1113] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:36:38 2022] GET /identity/v3/roles/0044294fcceb4b6ba0cb27567f2761d4 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:38.884787 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc797b71-bceb-4084-a9c2-6ab1923ff00c None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:39.000581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc797b71-bceb-4084-a9c2-6ab1923ff00c None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:39.034407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c455b-a0bf-4621-b438-c21d426a8dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:39.035292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c455b-a0bf-4621-b438-c21d426a8dfc None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:39.035570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c455b-a0bf-4621-b438-c21d426a8dfc None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:39.035844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c455b-a0bf-4621-b438-c21d426a8dfc None admin] PATH_INFO: `/v3/users/9b919963b3f34e388a04d1c17d6aa804` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:39.050046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c455b-a0bf-4621-b438-c21d426a8dfc None admin] RBAC: Authorizing `identity:get_user(user_id=9b919963b3f34e388a04d1c17d6aa804)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:39.051492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c455b-a0bf-4621-b438-c21d426a8dfc None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:39.053200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 586/1114] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:36:38 2022] GET /identity/v3/users/9b919963b3f34e388a04d1c17d6aa804 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:39.059699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08bca6-5b97-41bc-9560-76a5706dd5d3 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:39.062638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:39.170325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08bca6-5b97-41bc-9560-76a5706dd5d3 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:39.206356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0937858-1611-4331-a20e-a747a151d8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:39.207018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0937858-1611-4331-a20e-a747a151d8d7 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:39.207298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0937858-1611-4331-a20e-a747a151d8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:39.207570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0937858-1611-4331-a20e-a747a151d8d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:39.280210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0937858-1611-4331-a20e-a747a151d8d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:39.281742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0937858-1611-4331-a20e-a747a151d8d7 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:39.283789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:39.395945 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 529/1115] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:36:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:36:39.406088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c455b-a0bf-4621-b438-c21d426a8dfc None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:39.441430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:39.520816 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c455b-a0bf-4621-b438-c21d426a8dfc None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:39.555382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03455620-9263-48d8-8430-e10f5f0db9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:39.556246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03455620-9263-48d8-8430-e10f5f0db9f3 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:39.556562 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03455620-9263-48d8-8430-e10f5f0db9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:39.556849 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03455620-9263-48d8-8430-e10f5f0db9f3 None admin] PATH_INFO: `/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:39.558086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03455620-9263-48d8-8430-e10f5f0db9f3 None admin] RBAC: Authorizing `identity:get_project(project_id=c0fd7cd02b0645e1a208b86703a1aac8)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:39.559532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03455620-9263-48d8-8430-e10f5f0db9f3 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:39.561264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 587/1116] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:36:39 2022] GET /identity/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:39.567981 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0937858-1611-4331-a20e-a747a151d8d7 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:39.670616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0937858-1611-4331-a20e-a747a151d8d7 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:39.672198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:39.706096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1cebe-f682-4f4c-98ce-72a96f9c3a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:39.706660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1cebe-f682-4f4c-98ce-72a96f9c3a1b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:39.706937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1cebe-f682-4f4c-98ce-72a96f9c3a1b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:39.707205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1cebe-f682-4f4c-98ce-72a96f9c3a1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:39.708039 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1cebe-f682-4f4c-98ce-72a96f9c3a1b None admin] RBAC: Adding query filter params (role.id=0044294fcceb4b6ba0cb27567f2761d4, scope.project.id=c0fd7cd02b0645e1a208b86703a1aac8, user.id=9b919963b3f34e388a04d1c17d6aa804) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:39.708352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1cebe-f682-4f4c-98ce-72a96f9c3a1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:39.709836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1cebe-f682-4f4c-98ce-72a96f9c3a1b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:39.716166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 530/1117] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:36:39 2022] GET /identity/v3/role_assignments?user.id=9b919963b3f34e388a04d1c17d6aa804&scope.project.id=c0fd7cd02b0645e1a208b86703a1aac8&role.id=0044294fcceb4b6ba0cb27567f2761d4 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:41.864651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ebbda-3622-49f1-9eae-dd127545c04e None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:41.865199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ebbda-3622-49f1-9eae-dd127545c04e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:41.865659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ebbda-3622-49f1-9eae-dd127545c04e None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:41.868131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 588/1118] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:41.876041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b31b7-307d-4262-aac6-bbde34a388ff None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:41.876570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b31b7-307d-4262-aac6-bbde34a388ff None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:41.876999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b31b7-307d-4262-aac6-bbde34a388ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:41.903545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-807b31b7-307d-4262-aac6-bbde34a388ff None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:41.905616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-807b31b7-307d-4262-aac6-bbde34a388ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:42.015505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 531/1119] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:42.038714 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ebbda-3622-49f1-9eae-dd127545c04e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:42.136116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:42.148327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ebbda-3622-49f1-9eae-dd127545c04e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:42.187044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b14cf-7b33-4777-b850-45bbcc906185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:42.187901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b14cf-7b33-4777-b850-45bbcc906185 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:42.188189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b14cf-7b33-4777-b850-45bbcc906185 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:42.188471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b14cf-7b33-4777-b850-45bbcc906185 None admin] PATH_INFO: `/v3/roles/0044294fcceb4b6ba0cb27567f2761d4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:42.190007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b14cf-7b33-4777-b850-45bbcc906185 None admin] RBAC: Authorizing `identity:get_role(role_id=0044294fcceb4b6ba0cb27567f2761d4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:42.191395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b14cf-7b33-4777-b850-45bbcc906185 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:42.192803 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 589/1120] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:36:42 2022] GET /identity/v3/roles/0044294fcceb4b6ba0cb27567f2761d4 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:42.202590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b31b7-307d-4262-aac6-bbde34a388ff None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:42.301987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:42.309196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b31b7-307d-4262-aac6-bbde34a388ff None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:42.345201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f305b8-a443-4b3f-b1d8-ec05fd83b065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:42.346128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f305b8-a443-4b3f-b1d8-ec05fd83b065 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:42.346413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f305b8-a443-4b3f-b1d8-ec05fd83b065 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:42.346694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f305b8-a443-4b3f-b1d8-ec05fd83b065 None admin] PATH_INFO: `/v3/users/9b919963b3f34e388a04d1c17d6aa804` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:42.347963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f305b8-a443-4b3f-b1d8-ec05fd83b065 None admin] RBAC: Authorizing `identity:get_user(user_id=9b919963b3f34e388a04d1c17d6aa804)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:42.349465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f305b8-a443-4b3f-b1d8-ec05fd83b065 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:42.351171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 532/1121] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:36:42 2022] GET /identity/v3/users/9b919963b3f34e388a04d1c17d6aa804 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:42.361081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b14cf-7b33-4777-b850-45bbcc906185 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:42.474837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b14cf-7b33-4777-b850-45bbcc906185 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:42.512257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68147d90-35e2-4a0c-b2c4-f8b72adb2ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:42.513170 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147d90-35e2-4a0c-b2c4-f8b72adb2ec0 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:42.513553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147d90-35e2-4a0c-b2c4-f8b72adb2ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:42.513950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68147d90-35e2-4a0c-b2c4-f8b72adb2ec0 None admin] PATH_INFO: `/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:42.515510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68147d90-35e2-4a0c-b2c4-f8b72adb2ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=c0fd7cd02b0645e1a208b86703a1aac8)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:42.517043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68147d90-35e2-4a0c-b2c4-f8b72adb2ec0 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:42.518920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 590/1122] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:36:42 2022] GET /identity/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:42.526196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f305b8-a443-4b3f-b1d8-ec05fd83b065 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:42.631662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f305b8-a443-4b3f-b1d8-ec05fd83b065 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:42.671262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cee240-8c09-4dc2-9528-9c46ca610120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:42.672155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cee240-8c09-4dc2-9528-9c46ca610120 None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:42.672439 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cee240-8c09-4dc2-9528-9c46ca610120 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:42.672713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cee240-8c09-4dc2-9528-9c46ca610120 None admin] PATH_INFO: `/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8/users/9b919963b3f34e388a04d1c17d6aa804/roles/0044294fcceb4b6ba0cb27567f2761d4` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:42.675222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cee240-8c09-4dc2-9528-9c46ca610120 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0fd7cd02b0645e1a208b86703a1aac8, user_id=9b919963b3f34e388a04d1c17d6aa804, role_id=0044294fcceb4b6ba0cb27567f2761d4)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:42.676651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cee240-8c09-4dc2-9528-9c46ca610120 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:42.691746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 533/1123] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:36:42 2022] PUT /identity/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8/users/9b919963b3f34e388a04d1c17d6aa804/roles/0044294fcceb4b6ba0cb27567f2761d4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:36:44.847628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458dd7f-f73f-48d8-afe4-57747f5e9b58 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:44.848168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458dd7f-f73f-48d8-afe4-57747f5e9b58 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:44.848259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458dd7f-f73f-48d8-afe4-57747f5e9b58 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:44.849777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 591/1124] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:44.857226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539a29a-a951-43bf-b7a7-60afe735acb2 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:44.857805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539a29a-a951-43bf-b7a7-60afe735acb2 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:44.858223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539a29a-a951-43bf-b7a7-60afe735acb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:44.886116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-e539a29a-a951-43bf-b7a7-60afe735acb2 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:44.888112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e539a29a-a951-43bf-b7a7-60afe735acb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:44.918962 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:45.005506 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 534/1125] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:45.029244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458dd7f-f73f-48d8-afe4-57747f5e9b58 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:45.067150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:45.181002 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458dd7f-f73f-48d8-afe4-57747f5e9b58 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:45.219539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7769e6-20d0-43cb-a1d4-b868e7eab1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:45.220485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7769e6-20d0-43cb-a1d4-b868e7eab1c0 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:45.220809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7769e6-20d0-43cb-a1d4-b868e7eab1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:45.221142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7769e6-20d0-43cb-a1d4-b868e7eab1c0 None admin] PATH_INFO: `/v3/roles/0044294fcceb4b6ba0cb27567f2761d4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:45.222827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7769e6-20d0-43cb-a1d4-b868e7eab1c0 None admin] RBAC: Authorizing `identity:get_role(role_id=0044294fcceb4b6ba0cb27567f2761d4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:45.225080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7769e6-20d0-43cb-a1d4-b868e7eab1c0 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:45.225695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 592/1126] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:36:45 2022] GET /identity/v3/roles/0044294fcceb4b6ba0cb27567f2761d4 => generated 223 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:45.232836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539a29a-a951-43bf-b7a7-60afe735acb2 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:45.343702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539a29a-a951-43bf-b7a7-60afe735acb2 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:45.382131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5fbd9-3e91-496d-92d1-9c3fa0d1c580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:45.383401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5fbd9-3e91-496d-92d1-9c3fa0d1c580 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:45.383401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5fbd9-3e91-496d-92d1-9c3fa0d1c580 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:45.383625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5fbd9-3e91-496d-92d1-9c3fa0d1c580 None admin] PATH_INFO: `/v3/users/9b919963b3f34e388a04d1c17d6aa804` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:45.385328 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5fbd9-3e91-496d-92d1-9c3fa0d1c580 None admin] RBAC: Authorizing `identity:get_user(user_id=9b919963b3f34e388a04d1c17d6aa804)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:45.386832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5fbd9-3e91-496d-92d1-9c3fa0d1c580 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:45.388536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 535/1127] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:36:45 2022] GET /identity/v3/users/9b919963b3f34e388a04d1c17d6aa804 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:45.395036 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7769e6-20d0-43cb-a1d4-b868e7eab1c0 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:45.506768 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:45.515371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7769e6-20d0-43cb-a1d4-b868e7eab1c0 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:45.557193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec947d-2a38-462e-aaac-423a8749a85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:45.557881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec947d-2a38-462e-aaac-423a8749a85c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:45.558166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec947d-2a38-462e-aaac-423a8749a85c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:45.558447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec947d-2a38-462e-aaac-423a8749a85c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:45.630080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec947d-2a38-462e-aaac-423a8749a85c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:45.631516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec947d-2a38-462e-aaac-423a8749a85c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:45.750463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:45.758855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 593/1128] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:36:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:36:45.768085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5fbd9-3e91-496d-92d1-9c3fa0d1c580 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:45.793938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:45.902636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5fbd9-3e91-496d-92d1-9c3fa0d1c580 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:45.954896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d612d6-d81c-4548-81f6-8d521c1dcdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:45.956512 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d612d6-d81c-4548-81f6-8d521c1dcdfa None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:45.956990 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d612d6-d81c-4548-81f6-8d521c1dcdfa None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:45.957503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d612d6-d81c-4548-81f6-8d521c1dcdfa None admin] PATH_INFO: `/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:45.959931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d612d6-d81c-4548-81f6-8d521c1dcdfa None admin] RBAC: Authorizing `identity:get_project(project_id=c0fd7cd02b0645e1a208b86703a1aac8)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:45.963251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d612d6-d81c-4548-81f6-8d521c1dcdfa None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:45.966809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 536/1129] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:36:45 2022] GET /identity/v3/projects/c0fd7cd02b0645e1a208b86703a1aac8 => generated 311 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:45.973311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec947d-2a38-462e-aaac-423a8749a85c None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:46.117322 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec947d-2a38-462e-aaac-423a8749a85c None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:46.153351 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafb8f2-a699-43f4-a4b3-80e2d096ee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:46.154048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb8f2-a699-43f4-a4b3-80e2d096ee17 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:46.154336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb8f2-a699-43f4-a4b3-80e2d096ee17 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:46.154614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb8f2-a699-43f4-a4b3-80e2d096ee17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:46.155461 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafb8f2-a699-43f4-a4b3-80e2d096ee17 None admin] RBAC: Adding query filter params (role.id=0044294fcceb4b6ba0cb27567f2761d4, scope.project.id=c0fd7cd02b0645e1a208b86703a1aac8, user.id=9b919963b3f34e388a04d1c17d6aa804) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:46.155871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafb8f2-a699-43f4-a4b3-80e2d096ee17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:46.157237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafb8f2-a699-43f4-a4b3-80e2d096ee17 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:46.163829 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 594/1130] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:36:45 2022] GET /identity/v3/role_assignments?user.id=9b919963b3f34e388a04d1c17d6aa804&scope.project.id=c0fd7cd02b0645e1a208b86703a1aac8&role.id=0044294fcceb4b6ba0cb27567f2761d4 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:48.377949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad00185-5b06-4cca-902e-c1ba18d9555a None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:48.378695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad00185-5b06-4cca-902e-c1ba18d9555a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:48.378856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad00185-5b06-4cca-902e-c1ba18d9555a None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:48.380324 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 537/1131] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:48.387043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a35cc-dc1a-49d1-8bfb-0d9969d99918 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:48.387547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a35cc-dc1a-49d1-8bfb-0d9969d99918 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:48.387672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a35cc-dc1a-49d1-8bfb-0d9969d99918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:48.404804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-5b3a35cc-dc1a-49d1-8bfb-0d9969d99918 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:48.406396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5b3a35cc-dc1a-49d1-8bfb-0d9969d99918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:48.467851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:48.485762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 595/1132] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:36:48.510565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad00185-5b06-4cca-902e-c1ba18d9555a None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:48.612508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:48.625212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad00185-5b06-4cca-902e-c1ba18d9555a None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:48.660657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cfc44-9c88-479c-8aa9-29f4026b1a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:48.661557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cfc44-9c88-479c-8aa9-29f4026b1a9e None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:48.661840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cfc44-9c88-479c-8aa9-29f4026b1a9e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:48.662112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cfc44-9c88-479c-8aa9-29f4026b1a9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:48.663326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cfc44-9c88-479c-8aa9-29f4026b1a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:48.664770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cfc44-9c88-479c-8aa9-29f4026b1a9e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:48.666994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 538/1133] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:48.676437 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a35cc-dc1a-49d1-8bfb-0d9969d99918 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:48.848966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a35cc-dc1a-49d1-8bfb-0d9969d99918 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:48.887946 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9649f0-6a4b-4ccd-a137-5c2a2625d00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:48.888534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9649f0-6a4b-4ccd-a137-5c2a2625d00c None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:48.888811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9649f0-6a4b-4ccd-a137-5c2a2625d00c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:48.889084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9649f0-6a4b-4ccd-a137-5c2a2625d00c None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:48.889827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9649f0-6a4b-4ccd-a137-5c2a2625d00c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:48.891258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9649f0-6a4b-4ccd-a137-5c2a2625d00c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:48.919741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 596/1134] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:36:48 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:36:51.116689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e75ed2-6408-428a-9401-b146673b3111 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:51.117294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e75ed2-6408-428a-9401-b146673b3111 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:51.117434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e75ed2-6408-428a-9401-b146673b3111 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:51.120429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 539/1135] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:51.130105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d314209-19b6-4ffb-b308-242f40e4ac09 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:51.130909 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d314209-19b6-4ffb-b308-242f40e4ac09 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:51.131074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d314209-19b6-4ffb-b308-242f40e4ac09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:51.162093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-1d314209-19b6-4ffb-b308-242f40e4ac09 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:51.164176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1d314209-19b6-4ffb-b308-242f40e4ac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:51.233334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:51.301743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 597/1136] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:51.333594 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e75ed2-6408-428a-9401-b146673b3111 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:51.490606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e75ed2-6408-428a-9401-b146673b3111 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:51.532248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a952d30-8c09-4d0a-a984-458c6ae58472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:51.533136 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a952d30-8c09-4d0a-a984-458c6ae58472 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:51.533502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a952d30-8c09-4d0a-a984-458c6ae58472 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:51.533830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a952d30-8c09-4d0a-a984-458c6ae58472 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:51.535187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a952d30-8c09-4d0a-a984-458c6ae58472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:51.536729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a952d30-8c09-4d0a-a984-458c6ae58472 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:51.538603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 540/1137] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:36:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:51.545348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d314209-19b6-4ffb-b308-242f40e4ac09 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:51.673553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d314209-19b6-4ffb-b308-242f40e4ac09 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:51.731095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001bd3b-1f1b-420d-afbb-867ead47cc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:51.731794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001bd3b-1f1b-420d-afbb-867ead47cc71 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:51.732084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001bd3b-1f1b-420d-afbb-867ead47cc71 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:51.732366 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001bd3b-1f1b-420d-afbb-867ead47cc71 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:51.733121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001bd3b-1f1b-420d-afbb-867ead47cc71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:51.734899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001bd3b-1f1b-420d-afbb-867ead47cc71 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:51.766347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 598/1138] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:36:51 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:36:54.128636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3aa46-c307-456b-a707-628d74d78b40 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:54.129226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3aa46-c307-456b-a707-628d74d78b40 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:54.129226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3aa46-c307-456b-a707-628d74d78b40 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:54.130610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 541/1139] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:54.136698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127bcfa-b4c0-4cb0-91aa-7d354546c34a None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:54.137191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127bcfa-b4c0-4cb0-91aa-7d354546c34a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:54.137325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127bcfa-b4c0-4cb0-91aa-7d354546c34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:54.154279 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f127bcfa-b4c0-4cb0-91aa-7d354546c34a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:54.155518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f127bcfa-b4c0-4cb0-91aa-7d354546c34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:54.178252 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:54.231012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 599/1140] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:54.255607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3aa46-c307-456b-a707-628d74d78b40 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:54.284945 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:54.364689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3aa46-c307-456b-a707-628d74d78b40 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:54.405908 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbefdb-cef9-44f3-9328-98b25492e8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:54.407296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbefdb-cef9-44f3-9328-98b25492e8c6 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:54.407585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbefdb-cef9-44f3-9328-98b25492e8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:54.407867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbefdb-cef9-44f3-9328-98b25492e8c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:54.418795 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbefdb-cef9-44f3-9328-98b25492e8c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-c0dbefdb-cef9-44f3-9328-98b25492e8c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:36:54.426699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:54.427888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:54.428954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 542/1141] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:36:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:54.434853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127bcfa-b4c0-4cb0-91aa-7d354546c34a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:54.537231 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127bcfa-b4c0-4cb0-91aa-7d354546c34a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:54.572494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a873be7-13aa-4d12-a2a6-b03464e67def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:54.573209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a873be7-13aa-4d12-a2a6-b03464e67def None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:54.573536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a873be7-13aa-4d12-a2a6-b03464e67def None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:54.573822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a873be7-13aa-4d12-a2a6-b03464e67def None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:54.574592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a873be7-13aa-4d12-a2a6-b03464e67def None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:54.574907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a873be7-13aa-4d12-a2a6-b03464e67def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:54.576548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a873be7-13aa-4d12-a2a6-b03464e67def None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:54.585649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 600/1142] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:36:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:36:54.591603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbefdb-cef9-44f3-9328-98b25492e8c6 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:54.696281 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:54.721266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbefdb-cef9-44f3-9328-98b25492e8c6 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:54.779064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca38bf-10fa-42dc-a1a1-1bf098d98646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:54.779974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca38bf-10fa-42dc-a1a1-1bf098d98646 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:54.780256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca38bf-10fa-42dc-a1a1-1bf098d98646 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:54.780533 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca38bf-10fa-42dc-a1a1-1bf098d98646 None admin] PATH_INFO: `/v3/users/4ced6f2e96f2433892b1556d07f40aba` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:54.799888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca38bf-10fa-42dc-a1a1-1bf098d98646 None admin] RBAC: Authorizing `identity:get_user(user_id=4ced6f2e96f2433892b1556d07f40aba)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:54.801507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca38bf-10fa-42dc-a1a1-1bf098d98646 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:54.803426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 543/1143] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:36:54 2022] GET /identity/v3/users/4ced6f2e96f2433892b1556d07f40aba => generated 286 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:54.810543 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a873be7-13aa-4d12-a2a6-b03464e67def None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:54.844318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:54.951420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a873be7-13aa-4d12-a2a6-b03464e67def None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:54.986672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef643e7b-34c3-446d-a4a2-f30f829f6176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:54.987550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef643e7b-34c3-446d-a4a2-f30f829f6176 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:54.987870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef643e7b-34c3-446d-a4a2-f30f829f6176 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:54.988149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef643e7b-34c3-446d-a4a2-f30f829f6176 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:55.064611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef643e7b-34c3-446d-a4a2-f30f829f6176 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:55.066159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef643e7b-34c3-446d-a4a2-f30f829f6176 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:55.088593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:55.214653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 601/1144] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:36:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:36:55.222210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca38bf-10fa-42dc-a1a1-1bf098d98646 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:55.344703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca38bf-10fa-42dc-a1a1-1bf098d98646 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:55.387650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077df4a-6370-4538-9e6a-7a4eab92669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:55.388528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077df4a-6370-4538-9e6a-7a4eab92669b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:55.388814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077df4a-6370-4538-9e6a-7a4eab92669b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:55.389092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077df4a-6370-4538-9e6a-7a4eab92669b None admin] PATH_INFO: `/v3/projects/712fdac0977d4048813b5c6a161776b4` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:55.390390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077df4a-6370-4538-9e6a-7a4eab92669b None admin] RBAC: Authorizing `identity:get_project(project_id=712fdac0977d4048813b5c6a161776b4)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:55.391860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077df4a-6370-4538-9e6a-7a4eab92669b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:55.393736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 544/1145] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:36:55 2022] GET /identity/v3/projects/712fdac0977d4048813b5c6a161776b4 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:55.404378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef643e7b-34c3-446d-a4a2-f30f829f6176 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:55.555652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef643e7b-34c3-446d-a4a2-f30f829f6176 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:55.576990 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:55.597528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb5a4e-aebd-44b2-84da-edea1d1bda29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:55.598255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5a4e-aebd-44b2-84da-edea1d1bda29 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:55.598583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5a4e-aebd-44b2-84da-edea1d1bda29 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:55.598901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5a4e-aebd-44b2-84da-edea1d1bda29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:55.599791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb5a4e-aebd-44b2-84da-edea1d1bda29 None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=712fdac0977d4048813b5c6a161776b4, user.id=4ced6f2e96f2433892b1556d07f40aba) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:55.600144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb5a4e-aebd-44b2-84da-edea1d1bda29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:55.601664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb5a4e-aebd-44b2-84da-edea1d1bda29 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:55.608131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 602/1146] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:36:55 2022] GET /identity/v3/role_assignments?user.id=4ced6f2e96f2433892b1556d07f40aba&scope.project.id=712fdac0977d4048813b5c6a161776b4&role.id=85420c68300642aabfd730fb8fef8347 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:57.710973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f13086-ceee-430b-8ae7-ea48c7bec2cc None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:57.711503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f13086-ceee-430b-8ae7-ea48c7bec2cc None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:57.711503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f13086-ceee-430b-8ae7-ea48c7bec2cc None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:57.712997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 545/1147] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:36:57.718874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fc7fa-6b67-4247-ae5e-c80a0f3e871e None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:57.719401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fc7fa-6b67-4247-ae5e-c80a0f3e871e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:57.719541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fc7fa-6b67-4247-ae5e-c80a0f3e871e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:57.738187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-216fc7fa-6b67-4247-ae5e-c80a0f3e871e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:36:57.739456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-216fc7fa-6b67-4247-ae5e-c80a0f3e871e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:36:57.821119 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 603/1148] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:36:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:36:57.844374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f13086-ceee-430b-8ae7-ea48c7bec2cc None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:57.860904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:57.958479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f13086-ceee-430b-8ae7-ea48c7bec2cc None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:57.998052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b951b3-81f3-4f8f-84ef-df9b08837fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:57.998772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b951b3-81f3-4f8f-84ef-df9b08837fe0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:57.999061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b951b3-81f3-4f8f-84ef-df9b08837fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:57.999338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b951b3-81f3-4f8f-84ef-df9b08837fe0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:58.011173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b951b3-81f3-4f8f-84ef-df9b08837fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-48b951b3-81f3-4f8f-84ef-df9b08837fe0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:36:58.019181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:36:58.020195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:36:58.021282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 546/1149] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:36:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:36:58.028058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fc7fa-6b67-4247-ae5e-c80a0f3e871e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:58.141763 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fc7fa-6b67-4247-ae5e-c80a0f3e871e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:58.156457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:58.181762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863b1fb-95b9-4a5e-8795-af74e7a4fdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:58.182446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863b1fb-95b9-4a5e-8795-af74e7a4fdfe None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:58.182731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863b1fb-95b9-4a5e-8795-af74e7a4fdfe None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:58.183011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863b1fb-95b9-4a5e-8795-af74e7a4fdfe None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:58.183779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863b1fb-95b9-4a5e-8795-af74e7a4fdfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:36:58.184092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863b1fb-95b9-4a5e-8795-af74e7a4fdfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:58.185508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863b1fb-95b9-4a5e-8795-af74e7a4fdfe None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:58.194873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 604/1150] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:36:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:58.204888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b951b3-81f3-4f8f-84ef-df9b08837fe0 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:58.309187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:58.346710 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b951b3-81f3-4f8f-84ef-df9b08837fe0 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:58.386868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd03c5e-0ad8-4c61-81bb-d4cde11b4e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:58.387769 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd03c5e-0ad8-4c61-81bb-d4cde11b4e1f None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:58.388054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd03c5e-0ad8-4c61-81bb-d4cde11b4e1f None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:58.388330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd03c5e-0ad8-4c61-81bb-d4cde11b4e1f None admin] PATH_INFO: `/v3/users/4ced6f2e96f2433892b1556d07f40aba` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:58.389638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd03c5e-0ad8-4c61-81bb-d4cde11b4e1f None admin] RBAC: Authorizing `identity:get_user(user_id=4ced6f2e96f2433892b1556d07f40aba)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:58.391115 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd03c5e-0ad8-4c61-81bb-d4cde11b4e1f None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:58.392911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 547/1151] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:36:58 2022] GET /identity/v3/users/4ced6f2e96f2433892b1556d07f40aba => generated 286 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:58.402717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863b1fb-95b9-4a5e-8795-af74e7a4fdfe None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:58.514147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863b1fb-95b9-4a5e-8795-af74e7a4fdfe None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:58.563539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda73379-557a-453a-bf0b-aaa515573ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:58.564974 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73379-557a-453a-bf0b-aaa515573ebb None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:58.565554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73379-557a-453a-bf0b-aaa515573ebb None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:58.566225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73379-557a-453a-bf0b-aaa515573ebb None admin] PATH_INFO: `/v3/projects/712fdac0977d4048813b5c6a161776b4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:58.570793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda73379-557a-453a-bf0b-aaa515573ebb None admin] RBAC: Authorizing `identity:get_project(project_id=712fdac0977d4048813b5c6a161776b4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:58.574131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda73379-557a-453a-bf0b-aaa515573ebb None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:58.577770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 605/1152] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:36:58 2022] GET /identity/v3/projects/712fdac0977d4048813b5c6a161776b4 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:36:58.584782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd03c5e-0ad8-4c61-81bb-d4cde11b4e1f None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:36:58.701040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd03c5e-0ad8-4c61-81bb-d4cde11b4e1f None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:36:58.746867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1b39a-3584-46a6-8db4-7daac0f81dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:36:58.747805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1b39a-3584-46a6-8db4-7daac0f81dda None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:36:58.748090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1b39a-3584-46a6-8db4-7daac0f81dda None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:36:58.748367 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1b39a-3584-46a6-8db4-7daac0f81dda None admin] PATH_INFO: `/v3/projects/712fdac0977d4048813b5c6a161776b4/users/4ced6f2e96f2433892b1556d07f40aba/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:36:58.750485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1b39a-3584-46a6-8db4-7daac0f81dda None admin] RBAC: Authorizing `identity:create_grant(project_id=712fdac0977d4048813b5c6a161776b4, user_id=4ced6f2e96f2433892b1556d07f40aba, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:36:58.751932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1b39a-3584-46a6-8db4-7daac0f81dda None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:36:58.766339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:36:58.773567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 548/1153] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:36:58 2022] PUT /identity/v3/projects/712fdac0977d4048813b5c6a161776b4/users/4ced6f2e96f2433892b1556d07f40aba/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:37:01.000890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a498ca-b961-47b1-8bcf-7493e612b0de None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:01.001626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a498ca-b961-47b1-8bcf-7493e612b0de None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:01.001830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a498ca-b961-47b1-8bcf-7493e612b0de None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:01.003269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 606/1154] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:37:01.009516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3546f9-02a4-4aff-b5bf-84978c4be6bb None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:01.010343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3546f9-02a4-4aff-b5bf-84978c4be6bb None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:01.010526 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3546f9-02a4-4aff-b5bf-84978c4be6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:01.028649 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ac3546f9-02a4-4aff-b5bf-84978c4be6bb None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:37:01.029921 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ac3546f9-02a4-4aff-b5bf-84978c4be6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:37:01.108888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 549/1155] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:37:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:37:01.135059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a498ca-b961-47b1-8bcf-7493e612b0de None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:01.191048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:01.257173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a498ca-b961-47b1-8bcf-7493e612b0de None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:01.293080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07749fb4-fb83-4267-960d-3745109267d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:01.293918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07749fb4-fb83-4267-960d-3745109267d9 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:01.294198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07749fb4-fb83-4267-960d-3745109267d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:01.294473 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07749fb4-fb83-4267-960d-3745109267d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:01.306949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07749fb4-fb83-4267-960d-3745109267d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-07749fb4-fb83-4267-960d-3745109267d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:37:01.314750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:37:01.316462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:37:01.317303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 607/1156] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:37:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:37:01.326527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3546f9-02a4-4aff-b5bf-84978c4be6bb None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:01.459904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3546f9-02a4-4aff-b5bf-84978c4be6bb None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:01.496496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:01.503887 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a237bb0-28cd-4e4d-bcb2-347160589509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:01.504635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237bb0-28cd-4e4d-bcb2-347160589509 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:01.504960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237bb0-28cd-4e4d-bcb2-347160589509 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:01.505278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a237bb0-28cd-4e4d-bcb2-347160589509 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:01.506125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a237bb0-28cd-4e4d-bcb2-347160589509 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:37:01.506483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a237bb0-28cd-4e4d-bcb2-347160589509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:01.507983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a237bb0-28cd-4e4d-bcb2-347160589509 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:01.517898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 550/1157] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:37:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:37:01.527624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07749fb4-fb83-4267-960d-3745109267d9 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:01.651124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07749fb4-fb83-4267-960d-3745109267d9 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:01.652949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:01.691880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b954a3-d322-4ade-b6d3-91db71d7ec7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:01.692898 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b954a3-d322-4ade-b6d3-91db71d7ec7f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:01.693275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b954a3-d322-4ade-b6d3-91db71d7ec7f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:01.693690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b954a3-d322-4ade-b6d3-91db71d7ec7f None admin] PATH_INFO: `/v3/users/4ced6f2e96f2433892b1556d07f40aba` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:01.695076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b954a3-d322-4ade-b6d3-91db71d7ec7f None admin] RBAC: Authorizing `identity:get_user(user_id=4ced6f2e96f2433892b1556d07f40aba)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:01.696623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b954a3-d322-4ade-b6d3-91db71d7ec7f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:01.698390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 608/1158] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:37:01 2022] GET /identity/v3/users/4ced6f2e96f2433892b1556d07f40aba => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:37:01.704701 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a237bb0-28cd-4e4d-bcb2-347160589509 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:01.812284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a237bb0-28cd-4e4d-bcb2-347160589509 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:01.849737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0dc00-25c5-41d9-ac99-d6c11de6c8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:01.850403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0dc00-25c5-41d9-ac99-d6c11de6c8aa None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:01.850686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0dc00-25c5-41d9-ac99-d6c11de6c8aa None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:01.850961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0dc00-25c5-41d9-ac99-d6c11de6c8aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:01.914871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:01.926638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0dc00-25c5-41d9-ac99-d6c11de6c8aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:01.928205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0dc00-25c5-41d9-ac99-d6c11de6c8aa None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:02.038555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 551/1159] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:37:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 20:37:02.048282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b954a3-d322-4ade-b6d3-91db71d7ec7f None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:02.192599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b954a3-d322-4ade-b6d3-91db71d7ec7f None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:02.229103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0642261-6fdd-4842-ab09-7e0bc1d42dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:02.230554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0642261-6fdd-4842-ab09-7e0bc1d42dc7 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:02.230554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0642261-6fdd-4842-ab09-7e0bc1d42dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:02.230915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0642261-6fdd-4842-ab09-7e0bc1d42dc7 None admin] PATH_INFO: `/v3/projects/712fdac0977d4048813b5c6a161776b4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:02.232207 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0642261-6fdd-4842-ab09-7e0bc1d42dc7 None admin] RBAC: Authorizing `identity:get_project(project_id=712fdac0977d4048813b5c6a161776b4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:02.233724 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0642261-6fdd-4842-ab09-7e0bc1d42dc7 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:02.235579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 609/1160] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:37:02 2022] GET /identity/v3/projects/712fdac0977d4048813b5c6a161776b4 => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:37:02.242271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0dc00-25c5-41d9-ac99-d6c11de6c8aa None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:02.354806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:02.366439 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0dc00-25c5-41d9-ac99-d6c11de6c8aa None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:02.432218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9874f0a-4f5c-4b62-870f-b32904faafe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:02.432945 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9874f0a-4f5c-4b62-870f-b32904faafe1 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:02.433274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9874f0a-4f5c-4b62-870f-b32904faafe1 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:02.433632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9874f0a-4f5c-4b62-870f-b32904faafe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:02.434529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9874f0a-4f5c-4b62-870f-b32904faafe1 None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=712fdac0977d4048813b5c6a161776b4, user.id=4ced6f2e96f2433892b1556d07f40aba) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:37:02.434886 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9874f0a-4f5c-4b62-870f-b32904faafe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:02.436402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9874f0a-4f5c-4b62-870f-b32904faafe1 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:02.443858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 552/1161] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:37:02 2022] GET /identity/v3/role_assignments?user.id=4ced6f2e96f2433892b1556d07f40aba&scope.project.id=712fdac0977d4048813b5c6a161776b4&role.id=85420c68300642aabfd730fb8fef8347 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:37:04.592510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a39151-d690-414e-aac0-70e88cf9d46d None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:04.593082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a39151-d690-414e-aac0-70e88cf9d46d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:04.593082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a39151-d690-414e-aac0-70e88cf9d46d None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:04.594939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 610/1162] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:37:04.600938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d2517-2a76-4c5d-a1da-fb2a3695cb46 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:04.601488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d2517-2a76-4c5d-a1da-fb2a3695cb46 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:04.601635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d2517-2a76-4c5d-a1da-fb2a3695cb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:04.619914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-729d2517-2a76-4c5d-a1da-fb2a3695cb46 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:37:04.621117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-729d2517-2a76-4c5d-a1da-fb2a3695cb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:37:04.692348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 553/1163] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:37:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:37:04.715663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a39151-d690-414e-aac0-70e88cf9d46d None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:04.765501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:04.832050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a39151-d690-414e-aac0-70e88cf9d46d None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:04.869149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d60a6-cda0-48ba-a2c2-b164f3f152c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:04.869831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d60a6-cda0-48ba-a2c2-b164f3f152c9 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:04.870115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d60a6-cda0-48ba-a2c2-b164f3f152c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:04.870388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d60a6-cda0-48ba-a2c2-b164f3f152c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:04.942337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d60a6-cda0-48ba-a2c2-b164f3f152c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:04.943873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d60a6-cda0-48ba-a2c2-b164f3f152c9 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:04.988882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:05.063993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 611/1164] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:37:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:37:05.070364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d2517-2a76-4c5d-a1da-fb2a3695cb46 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:05.160684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:05.178623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d2517-2a76-4c5d-a1da-fb2a3695cb46 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:05.215767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df514f-e14d-4d62-adc3-787124511c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:05.216725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df514f-e14d-4d62-adc3-787124511c5b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:05.217011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df514f-e14d-4d62-adc3-787124511c5b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:05.217282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df514f-e14d-4d62-adc3-787124511c5b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:05.225762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df514f-e14d-4d62-adc3-787124511c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:05.227254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df514f-e14d-4d62-adc3-787124511c5b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-a9df514f-e14d-4d62-adc3-787124511c5b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:37:05.242681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:37:05.244528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:37:05.246145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 554/1165] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jul 19 20:37:05 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:37:05.250315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d60a6-cda0-48ba-a2c2-b164f3f152c9 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:05.363286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d60a6-cda0-48ba-a2c2-b164f3f152c9 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:05.406771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b5bdf-dd68-4823-aad2-c49c13ee0f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:05.407510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b5bdf-dd68-4823-aad2-c49c13ee0f24 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:05.407837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b5bdf-dd68-4823-aad2-c49c13ee0f24 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:05.408154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b5bdf-dd68-4823-aad2-c49c13ee0f24 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:05.408939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b5bdf-dd68-4823-aad2-c49c13ee0f24 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:37:05.409291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b5bdf-dd68-4823-aad2-c49c13ee0f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:05.410739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b5bdf-dd68-4823-aad2-c49c13ee0f24 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:05.419348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 612/1166] 10.222.0.70 () {56 vars in 1115 bytes} [Tue Jul 19 20:37:05 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:37:07.554676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9383c0-a440-46d2-8344-3f49e42a4342 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:07.555268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9383c0-a440-46d2-8344-3f49e42a4342 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:07.555268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9383c0-a440-46d2-8344-3f49e42a4342 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:07.556629 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 555/1167] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:37:07.563167 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de448f35-3daa-43f7-a0f3-5493607c8dd3 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:07.563755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de448f35-3daa-43f7-a0f3-5493607c8dd3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:07.563755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de448f35-3daa-43f7-a0f3-5493607c8dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:07.569982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:07.584778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-de448f35-3daa-43f7-a0f3-5493607c8dd3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:37:07.586043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-de448f35-3daa-43f7-a0f3-5493607c8dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:37:07.661206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 613/1168] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:37:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:37:07.684588 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9383c0-a440-46d2-8344-3f49e42a4342 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:07.795876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9383c0-a440-46d2-8344-3f49e42a4342 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:07.832831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47632d43-3f4b-4b3f-8117-cd51259c01e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:07.833526 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47632d43-3f4b-4b3f-8117-cd51259c01e6 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:07.833809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47632d43-3f4b-4b3f-8117-cd51259c01e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:07.834082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47632d43-3f4b-4b3f-8117-cd51259c01e6 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:07.834850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47632d43-3f4b-4b3f-8117-cd51259c01e6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:07.836253 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47632d43-3f4b-4b3f-8117-cd51259c01e6 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:07.855150 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 556/1169] 10.222.0.70 () {60 vars in 1138 bytes} [Tue Jul 19 20:37:07 2022] POST /identity/v3/domains => generated 267 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:37:10.014920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8f3b-fc8e-408e-98c0-1e36dee3e0ef None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:10.015866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8f3b-fc8e-408e-98c0-1e36dee3e0ef None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:10.015866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8f3b-fc8e-408e-98c0-1e36dee3e0ef None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:10.017607 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 614/1170] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:37:10.023890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b4ba4-df48-4466-9189-7747e5b365d5 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:10.024444 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b4ba4-df48-4466-9189-7747e5b365d5 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:10.024444 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b4ba4-df48-4466-9189-7747e5b365d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:10.042158 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:10.050561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-d71b4ba4-df48-4466-9189-7747e5b365d5 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:37:10.052200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d71b4ba4-df48-4466-9189-7747e5b365d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:37:10.138160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 557/1171] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:37:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:37:10.169275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a8f3b-fc8e-408e-98c0-1e36dee3e0ef None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:10.335969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a8f3b-fc8e-408e-98c0-1e36dee3e0ef None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:10.369090 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:10.376932 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eff2e0-0b44-4634-9247-95eb19de4555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:10.377852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff2e0-0b44-4634-9247-95eb19de4555 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:10.378129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff2e0-0b44-4634-9247-95eb19de4555 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:10.378399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff2e0-0b44-4634-9247-95eb19de4555 None admin] PATH_INFO: `/v3/domains/65278a49841d4bcdb3092b5ad39f4534` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:10.391025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eff2e0-0b44-4634-9247-95eb19de4555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=65278a49841d4bcdb3092b5ad39f4534)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:10.392479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eff2e0-0b44-4634-9247-95eb19de4555 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:10.394501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 615/1172] 10.222.0.70 () {56 vars in 1183 bytes} [Tue Jul 19 20:37:10 2022] GET /identity/v3/domains/65278a49841d4bcdb3092b5ad39f4534 => generated 267 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:37:10.401202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b4ba4-df48-4466-9189-7747e5b365d5 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:10.546270 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b4ba4-df48-4466-9189-7747e5b365d5 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:10.603857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef56317-08aa-48c4-a321-8e7f0ec31e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:10.604438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef56317-08aa-48c4-a321-8e7f0ec31e13 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:10.604716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef56317-08aa-48c4-a321-8e7f0ec31e13 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:10.604989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef56317-08aa-48c4-a321-8e7f0ec31e13 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:10.605745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef56317-08aa-48c4-a321-8e7f0ec31e13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:10.607202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef56317-08aa-48c4-a321-8e7f0ec31e13 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:10.611936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:10.642751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 558/1173] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:37:10 2022] POST /identity/v3/projects => generated 361 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:37:13.026186 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c476e3-4073-4e8c-891e-5f2828c2e867 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:13.027003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c476e3-4073-4e8c-891e-5f2828c2e867 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:13.027003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c476e3-4073-4e8c-891e-5f2828c2e867 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:13.028142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 616/1174] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:37:13.034131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aa7b6-df2f-4955-ad05-016087e901c3 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:13.034929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aa7b6-df2f-4955-ad05-016087e901c3 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:13.035084 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aa7b6-df2f-4955-ad05-016087e901c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:13.053991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-4f0aa7b6-df2f-4955-ad05-016087e901c3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:37:13.055204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4f0aa7b6-df2f-4955-ad05-016087e901c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:37:13.133513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 559/1175] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:37:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:37:13.159049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c476e3-4073-4e8c-891e-5f2828c2e867 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:13.327621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c476e3-4073-4e8c-891e-5f2828c2e867 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:13.388757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9da1e-030e-4993-baef-8adbd824391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:13.390325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9da1e-030e-4993-baef-8adbd824391d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:13.390848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9da1e-030e-4993-baef-8adbd824391d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:13.391351 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9da1e-030e-4993-baef-8adbd824391d None admin] PATH_INFO: `/v3/domains/65278a49841d4bcdb3092b5ad39f4534` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:13.393600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9da1e-030e-4993-baef-8adbd824391d None admin] RBAC: Authorizing `identity:get_domain(domain_id=65278a49841d4bcdb3092b5ad39f4534)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:13.396385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9da1e-030e-4993-baef-8adbd824391d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:13.399506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 617/1176] 10.222.0.70 () {56 vars in 1183 bytes} [Tue Jul 19 20:37:13 2022] GET /identity/v3/domains/65278a49841d4bcdb3092b5ad39f4534 => generated 267 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:37:13.406343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aa7b6-df2f-4955-ad05-016087e901c3 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:13.515862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aa7b6-df2f-4955-ad05-016087e901c3 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:13.534917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:13.553713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc961a15-7883-4b37-89c7-5c5878041876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:13.554411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc961a15-7883-4b37-89c7-5c5878041876 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:13.554690 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc961a15-7883-4b37-89c7-5c5878041876 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:13.554961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc961a15-7883-4b37-89c7-5c5878041876 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:13.555721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc961a15-7883-4b37-89c7-5c5878041876 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:13.557229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc961a15-7883-4b37-89c7-5c5878041876 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:13.580048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 560/1177] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:37:13 2022] POST /identity/v3/users => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:37:15.769255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41204c81-5e04-4183-9c78-6bb1faa5bf79 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:15.769861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41204c81-5e04-4183-9c78-6bb1faa5bf79 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:15.770192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41204c81-5e04-4183-9c78-6bb1faa5bf79 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:15.772702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 618/1178] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:37:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:37:15.779096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa846977-1f5f-4354-9bee-8b626308d125 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:15.779664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa846977-1f5f-4354-9bee-8b626308d125 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:15.779664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa846977-1f5f-4354-9bee-8b626308d125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:15.797533 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-aa846977-1f5f-4354-9bee-8b626308d125 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:37:15.798758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-aa846977-1f5f-4354-9bee-8b626308d125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:37:15.871997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 561/1179] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:37:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:37:15.896691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41204c81-5e04-4183-9c78-6bb1faa5bf79 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:15.964751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:16.010788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41204c81-5e04-4183-9c78-6bb1faa5bf79 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:16.052510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b6d4a-de0a-4f05-983b-ddf7587a08e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:16.053311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b6d4a-de0a-4f05-983b-ddf7587a08e4 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:16.053631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b6d4a-de0a-4f05-983b-ddf7587a08e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:16.053914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b6d4a-de0a-4f05-983b-ddf7587a08e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:16.066661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b6d4a-de0a-4f05-983b-ddf7587a08e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-9b6b6d4a-de0a-4f05-983b-ddf7587a08e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:37:16.074800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:37:16.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:37:16.078733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 619/1180] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:37:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:37:16.084327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa846977-1f5f-4354-9bee-8b626308d125 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:16.194938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa846977-1f5f-4354-9bee-8b626308d125 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:16.198147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:16.235898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d693a-d357-4d8a-8bb4-21662bcfae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:16.236588 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d693a-d357-4d8a-8bb4-21662bcfae09 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:16.236871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d693a-d357-4d8a-8bb4-21662bcfae09 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:16.237143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d693a-d357-4d8a-8bb4-21662bcfae09 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:16.237946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d693a-d357-4d8a-8bb4-21662bcfae09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:37:16.238248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d693a-d357-4d8a-8bb4-21662bcfae09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:16.239662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d693a-d357-4d8a-8bb4-21662bcfae09 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:16.249891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 562/1181] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:37:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:37:16.256085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b6d4a-de0a-4f05-983b-ddf7587a08e4 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:16.387190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b6d4a-de0a-4f05-983b-ddf7587a08e4 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:16.409269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:16.425995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955701d-2f29-49a8-83d7-9883d4b45127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:16.427196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955701d-2f29-49a8-83d7-9883d4b45127 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:16.427538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955701d-2f29-49a8-83d7-9883d4b45127 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:16.427913 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955701d-2f29-49a8-83d7-9883d4b45127 None admin] PATH_INFO: `/v3/users/8065f9d14fe542e9882a608342be289f` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:16.444282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955701d-2f29-49a8-83d7-9883d4b45127 None admin] RBAC: Authorizing `identity:get_user(user_id=8065f9d14fe542e9882a608342be289f)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:16.445776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955701d-2f29-49a8-83d7-9883d4b45127 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:16.447594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 620/1182] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:37:16 2022] GET /identity/v3/users/8065f9d14fe542e9882a608342be289f => generated 311 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:37:16.454081 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d693a-d357-4d8a-8bb4-21662bcfae09 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:16.563593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d693a-d357-4d8a-8bb4-21662bcfae09 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:16.599695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a37d46-7d5e-4262-8401-05789ef9bda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:16.600357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a37d46-7d5e-4262-8401-05789ef9bda8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:16.600637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a37d46-7d5e-4262-8401-05789ef9bda8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:16.600909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a37d46-7d5e-4262-8401-05789ef9bda8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:16.637093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:16.677329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a37d46-7d5e-4262-8401-05789ef9bda8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:16.678981 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a37d46-7d5e-4262-8401-05789ef9bda8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:16.800591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 563/1183] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:37:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:37:16.809705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955701d-2f29-49a8-83d7-9883d4b45127 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:16.956583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955701d-2f29-49a8-83d7-9883d4b45127 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:16.995734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebaf12-3341-45ee-bd35-6c0e5594d057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:16.996604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebaf12-3341-45ee-bd35-6c0e5594d057 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:16.996891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebaf12-3341-45ee-bd35-6c0e5594d057 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:16.997171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebaf12-3341-45ee-bd35-6c0e5594d057 None admin] PATH_INFO: `/v3/projects/481cb685137d4a599ac1056732364fd8` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:16.998468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebaf12-3341-45ee-bd35-6c0e5594d057 None admin] RBAC: Authorizing `identity:get_project(project_id=481cb685137d4a599ac1056732364fd8)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:17.000089 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebaf12-3341-45ee-bd35-6c0e5594d057 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:17.001897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 621/1184] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:37:16 2022] GET /identity/v3/projects/481cb685137d4a599ac1056732364fd8 => generated 361 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:37:17.008334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a37d46-7d5e-4262-8401-05789ef9bda8 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:17.092226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:17.131058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a37d46-7d5e-4262-8401-05789ef9bda8 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:17.170501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b29064-ba2d-4161-a1bb-a3a1cd0daadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:17.171161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29064-ba2d-4161-a1bb-a3a1cd0daadb None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:17.171443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29064-ba2d-4161-a1bb-a3a1cd0daadb None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:17.171719 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29064-ba2d-4161-a1bb-a3a1cd0daadb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:17.172569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b29064-ba2d-4161-a1bb-a3a1cd0daadb None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=481cb685137d4a599ac1056732364fd8, user.id=8065f9d14fe542e9882a608342be289f) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:37:17.172892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b29064-ba2d-4161-a1bb-a3a1cd0daadb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:17.174460 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b29064-ba2d-4161-a1bb-a3a1cd0daadb None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:17.185691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 564/1185] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:37:17 2022] GET /identity/v3/role_assignments?user.id=8065f9d14fe542e9882a608342be289f&scope.project.id=481cb685137d4a599ac1056732364fd8&role.id=85420c68300642aabfd730fb8fef8347 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:37:19.308931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b3a0d-c3c4-4cd7-af2c-d35502eaaf8b None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:19.309649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b3a0d-c3c4-4cd7-af2c-d35502eaaf8b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:19.309649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b3a0d-c3c4-4cd7-af2c-d35502eaaf8b None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:19.310900 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 622/1186] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:37:19.317681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebd118-800b-411b-9654-621c9b6037b4 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:19.318255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebd118-800b-411b-9654-621c9b6037b4 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:19.318668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebd118-800b-411b-9654-621c9b6037b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:19.346626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-d6ebd118-800b-411b-9654-621c9b6037b4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:37:19.350820 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d6ebd118-800b-411b-9654-621c9b6037b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:37:19.428650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 565/1187] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:37:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:37:19.452949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b3a0d-c3c4-4cd7-af2c-d35502eaaf8b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:19.506893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:19.567529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b3a0d-c3c4-4cd7-af2c-d35502eaaf8b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:19.606534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494fb4f-85a7-4ca2-83ec-063a14bc2e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:19.607337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494fb4f-85a7-4ca2-83ec-063a14bc2e0e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:19.607623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494fb4f-85a7-4ca2-83ec-063a14bc2e0e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:19.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494fb4f-85a7-4ca2-83ec-063a14bc2e0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:19.619876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494fb4f-85a7-4ca2-83ec-063a14bc2e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-3494fb4f-85a7-4ca2-83ec-063a14bc2e0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:37:19.627840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:37:19.629680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:37:19.630848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 623/1188] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:37:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:37:19.636132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebd118-800b-411b-9654-621c9b6037b4 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:19.720734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:19.769247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebd118-800b-411b-9654-621c9b6037b4 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:19.806736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d113f-c67b-4d10-9b0e-80e5d31aee9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:19.807415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d113f-c67b-4d10-9b0e-80e5d31aee9d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:19.807695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d113f-c67b-4d10-9b0e-80e5d31aee9d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:19.807967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d113f-c67b-4d10-9b0e-80e5d31aee9d None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:19.808736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d113f-c67b-4d10-9b0e-80e5d31aee9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:37:19.809043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d113f-c67b-4d10-9b0e-80e5d31aee9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:19.810537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d113f-c67b-4d10-9b0e-80e5d31aee9d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:19.821116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 566/1189] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:37:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:37:19.828210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494fb4f-85a7-4ca2-83ec-063a14bc2e0e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:19.938373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494fb4f-85a7-4ca2-83ec-063a14bc2e0e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:19.946305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:19.978530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a64a96-b601-4636-8103-ce7e0c8dbc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:19.979394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a64a96-b601-4636-8103-ce7e0c8dbc61 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:19.979676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a64a96-b601-4636-8103-ce7e0c8dbc61 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:19.979952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a64a96-b601-4636-8103-ce7e0c8dbc61 None admin] PATH_INFO: `/v3/users/8065f9d14fe542e9882a608342be289f` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:19.981167 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a64a96-b601-4636-8103-ce7e0c8dbc61 None admin] RBAC: Authorizing `identity:get_user(user_id=8065f9d14fe542e9882a608342be289f)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:19.982633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a64a96-b601-4636-8103-ce7e0c8dbc61 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:19.984857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 624/1190] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:37:19 2022] GET /identity/v3/users/8065f9d14fe542e9882a608342be289f => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:37:19.995408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d113f-c67b-4d10-9b0e-80e5d31aee9d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:20.174961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d113f-c67b-4d10-9b0e-80e5d31aee9d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:20.219384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:20.239933 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29458232-5d64-4074-b040-d8c0dd8cbbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:20.241467 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29458232-5d64-4074-b040-d8c0dd8cbbc1 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:20.241974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29458232-5d64-4074-b040-d8c0dd8cbbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:20.242478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29458232-5d64-4074-b040-d8c0dd8cbbc1 None admin] PATH_INFO: `/v3/projects/481cb685137d4a599ac1056732364fd8` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:20.245371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29458232-5d64-4074-b040-d8c0dd8cbbc1 None admin] RBAC: Authorizing `identity:get_project(project_id=481cb685137d4a599ac1056732364fd8)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:20.247987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29458232-5d64-4074-b040-d8c0dd8cbbc1 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:20.251407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 567/1191] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:37:19 2022] GET /identity/v3/projects/481cb685137d4a599ac1056732364fd8 => generated 361 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:37:20.258003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a64a96-b601-4636-8103-ce7e0c8dbc61 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:20.377881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a64a96-b601-4636-8103-ce7e0c8dbc61 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:20.416871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebba19-e9c6-44df-8fa4-36ba50eddf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:20.417797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebba19-e9c6-44df-8fa4-36ba50eddf20 None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:20.418138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebba19-e9c6-44df-8fa4-36ba50eddf20 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:20.418417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebba19-e9c6-44df-8fa4-36ba50eddf20 None admin] PATH_INFO: `/v3/projects/481cb685137d4a599ac1056732364fd8/users/8065f9d14fe542e9882a608342be289f/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:20.420554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebba19-e9c6-44df-8fa4-36ba50eddf20 None admin] RBAC: Authorizing `identity:create_grant(project_id=481cb685137d4a599ac1056732364fd8, user_id=8065f9d14fe542e9882a608342be289f, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:20.422042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebba19-e9c6-44df-8fa4-36ba50eddf20 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:20.438337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 625/1192] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:37:20 2022] PUT /identity/v3/projects/481cb685137d4a599ac1056732364fd8/users/8065f9d14fe542e9882a608342be289f/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:37:22.539821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e68fc-57d8-4a59-9797-f4779c65e5ff None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:22.540906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e68fc-57d8-4a59-9797-f4779c65e5ff None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:22.540906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e68fc-57d8-4a59-9797-f4779c65e5ff None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:22.542449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 568/1193] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:37:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:37:22.548715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5651772-ec24-4b4f-822c-4556eab3cb3a None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:22.549311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5651772-ec24-4b4f-822c-4556eab3cb3a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:22.549311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5651772-ec24-4b4f-822c-4556eab3cb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:22.567933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f5651772-ec24-4b4f-822c-4556eab3cb3a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:37:22.569135 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f5651772-ec24-4b4f-822c-4556eab3cb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:37:22.600601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:22.649044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 626/1194] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:37:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:37:22.674499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e68fc-57d8-4a59-9797-f4779c65e5ff None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:22.785578 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e68fc-57d8-4a59-9797-f4779c65e5ff None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:22.822261 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca6806-9ae7-4dc8-a2ba-ceca2ccc5a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:22.823055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6806-9ae7-4dc8-a2ba-ceca2ccc5a44 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:22.823335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6806-9ae7-4dc8-a2ba-ceca2ccc5a44 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:22.823607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6806-9ae7-4dc8-a2ba-ceca2ccc5a44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:22.834496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca6806-9ae7-4dc8-a2ba-ceca2ccc5a44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-d4ca6806-9ae7-4dc8-a2ba-ceca2ccc5a44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:37:22.842777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:37:22.844415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:37:22.845306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 569/1195] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:37:22.850381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5651772-ec24-4b4f-822c-4556eab3cb3a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:22.959161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5651772-ec24-4b4f-822c-4556eab3cb3a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:22.997605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ad4ed-05dc-4404-abaa-1e199bd69ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:22.998331 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ad4ed-05dc-4404-abaa-1e199bd69ee6 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:22.998653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ad4ed-05dc-4404-abaa-1e199bd69ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:22.998972 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ad4ed-05dc-4404-abaa-1e199bd69ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:22.999814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ad4ed-05dc-4404-abaa-1e199bd69ee6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:37:23.000167 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ad4ed-05dc-4404-abaa-1e199bd69ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:23.001694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ad4ed-05dc-4404-abaa-1e199bd69ee6 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:23.013392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 627/1196] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:37:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:37:23.019651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca6806-9ae7-4dc8-a2ba-ceca2ccc5a44 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:23.073582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:23.142072 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca6806-9ae7-4dc8-a2ba-ceca2ccc5a44 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:23.179796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cce65d-c0ba-4a5f-8a4c-78787684f493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:23.180680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce65d-c0ba-4a5f-8a4c-78787684f493 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:23.180963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce65d-c0ba-4a5f-8a4c-78787684f493 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:23.181238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce65d-c0ba-4a5f-8a4c-78787684f493 None admin] PATH_INFO: `/v3/users/8065f9d14fe542e9882a608342be289f` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:23.182488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cce65d-c0ba-4a5f-8a4c-78787684f493 None admin] RBAC: Authorizing `identity:get_user(user_id=8065f9d14fe542e9882a608342be289f)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:23.183929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cce65d-c0ba-4a5f-8a4c-78787684f493 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:23.185646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 570/1197] 10.222.0.70 () {56 vars in 1177 bytes} [Tue Jul 19 20:37:23 2022] GET /identity/v3/users/8065f9d14fe542e9882a608342be289f => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:37:23.191916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ad4ed-05dc-4404-abaa-1e199bd69ee6 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:23.224202 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:23.300270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ad4ed-05dc-4404-abaa-1e199bd69ee6 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:23.336624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209bc69-4836-489d-b89b-755b485882de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:23.337840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209bc69-4836-489d-b89b-755b485882de None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:23.338126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209bc69-4836-489d-b89b-755b485882de None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:23.338402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209bc69-4836-489d-b89b-755b485882de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:23.425723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209bc69-4836-489d-b89b-755b485882de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:23.427248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209bc69-4836-489d-b89b-755b485882de None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:23.455991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:23.534675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 628/1198] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:37:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:37:23.544147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cce65d-c0ba-4a5f-8a4c-78787684f493 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:23.656009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cce65d-c0ba-4a5f-8a4c-78787684f493 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:23.674875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:23.694279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b6fd9-476c-4912-a626-3b7d0a440b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:23.695155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6fd9-476c-4912-a626-3b7d0a440b71 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:23.695435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6fd9-476c-4912-a626-3b7d0a440b71 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:23.695707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6fd9-476c-4912-a626-3b7d0a440b71 None admin] PATH_INFO: `/v3/projects/481cb685137d4a599ac1056732364fd8` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:23.697504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b6fd9-476c-4912-a626-3b7d0a440b71 None admin] RBAC: Authorizing `identity:get_project(project_id=481cb685137d4a599ac1056732364fd8)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:23.698945 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b6fd9-476c-4912-a626-3b7d0a440b71 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:23.701083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 571/1199] 10.222.0.70 () {56 vars in 1186 bytes} [Tue Jul 19 20:37:23 2022] GET /identity/v3/projects/481cb685137d4a599ac1056732364fd8 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:37:23.710708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209bc69-4836-489d-b89b-755b485882de None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:37:23.820975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209bc69-4836-489d-b89b-755b485882de None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:37:23.848090 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:23.860438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b994e-16ea-46da-b909-122619c3ca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:37:23.861632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b994e-16ea-46da-b909-122619c3ca8e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:37:23.862133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b994e-16ea-46da-b909-122619c3ca8e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:37:23.862633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b994e-16ea-46da-b909-122619c3ca8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:37:23.864225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b994e-16ea-46da-b909-122619c3ca8e None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=481cb685137d4a599ac1056732364fd8, user.id=8065f9d14fe542e9882a608342be289f) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:37:23.864773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b994e-16ea-46da-b909-122619c3ca8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:37:23.867422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b994e-16ea-46da-b909-122619c3ca8e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:37:23.879192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 629/1200] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:37:23 2022] GET /identity/v3/role_assignments?user.id=8065f9d14fe542e9882a608342be289f&scope.project.id=481cb685137d4a599ac1056732364fd8&role.id=85420c68300642aabfd730fb8fef8347 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:37:33.689878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=7 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:37:33.874014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:25.920092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebad91-1b98-40b1-91a7-cb4712577523 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:25.920092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebad91-1b98-40b1-91a7-cb4712577523 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:25.920092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebad91-1b98-40b1-91a7-cb4712577523 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:25.920092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 572/1201] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:38:25.926681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d20ec-bfcd-43c3-b382-0b84e377bcc5 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:25.927572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d20ec-bfcd-43c3-b382-0b84e377bcc5 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:25.927572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d20ec-bfcd-43c3-b382-0b84e377bcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:25.947915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-002d20ec-bfcd-43c3-b382-0b84e377bcc5 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:38:25.949326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-002d20ec-bfcd-43c3-b382-0b84e377bcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:38:26.064563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 630/1202] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:38:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:38:26.098790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebad91-1b98-40b1-91a7-cb4712577523 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:26.269929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebad91-1b98-40b1-91a7-cb4712577523 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:26.333792 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cd34e-56fc-4676-b3cc-ad614e7350b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:26.335027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cd34e-56fc-4676-b3cc-ad614e7350b2 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:26.335421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cd34e-56fc-4676-b3cc-ad614e7350b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:26.335894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cd34e-56fc-4676-b3cc-ad614e7350b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:26.355061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cd34e-56fc-4676-b3cc-ad614e7350b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:26.359174 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cd34e-56fc-4676-b3cc-ad614e7350b2 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-966cd34e-56fc-4676-b3cc-ad614e7350b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:26.390883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:26.397393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:26.399087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:26.399087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:26.399087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:26.399087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:26.399087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 573/1203] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:26.403636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d20ec-bfcd-43c3-b382-0b84e377bcc5 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:26.563335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d20ec-bfcd-43c3-b382-0b84e377bcc5 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:26.615753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:26.629632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184de4f-1445-43ea-9af6-7c1acb696480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:26.630354 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184de4f-1445-43ea-9af6-7c1acb696480 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:26.630575 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184de4f-1445-43ea-9af6-7c1acb696480 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:26.630865 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184de4f-1445-43ea-9af6-7c1acb696480 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:26.631615 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184de4f-1445-43ea-9af6-7c1acb696480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:26.631964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184de4f-1445-43ea-9af6-7c1acb696480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:26.633532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184de4f-1445-43ea-9af6-7c1acb696480 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:26.647832 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 631/1204] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:26.657135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cd34e-56fc-4676-b3cc-ad614e7350b2 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:26.765472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:26.864369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cd34e-56fc-4676-b3cc-ad614e7350b2 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:26.924330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd17e55-62ab-4494-abe2-1100d53d19ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:26.925341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd17e55-62ab-4494-abe2-1100d53d19ed None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:26.925341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd17e55-62ab-4494-abe2-1100d53d19ed None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:26.925597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd17e55-62ab-4494-abe2-1100d53d19ed None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:26.926341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd17e55-62ab-4494-abe2-1100d53d19ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:26.927891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd17e55-62ab-4494-abe2-1100d53d19ed None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:26.965392 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 574/1205] 10.222.0.70 () {60 vars in 1132 bytes} [Tue Jul 19 20:38:26 2022] POST /identity/v3/users => generated 251 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:38:29.295323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b52f4b-c5de-412a-b30a-539a84652f83 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:29.298130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b52f4b-c5de-412a-b30a-539a84652f83 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:29.298130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b52f4b-c5de-412a-b30a-539a84652f83 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:29.299518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 632/1206] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:38:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:38:29.307455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b96f6-cc29-46a4-a08c-95b2a8c9b205 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:29.307455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b96f6-cc29-46a4-a08c-95b2a8c9b205 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:29.307455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b96f6-cc29-46a4-a08c-95b2a8c9b205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:29.326757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c01b96f6-cc29-46a4-a08c-95b2a8c9b205 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:38:29.328301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c01b96f6-cc29-46a4-a08c-95b2a8c9b205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:38:29.415927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 575/1207] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:38:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:38:29.442519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b52f4b-c5de-412a-b30a-539a84652f83 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:29.595097 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b52f4b-c5de-412a-b30a-539a84652f83 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:29.635476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700de1b2-ef62-46bf-ab05-5068136243a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:29.636297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700de1b2-ef62-46bf-ab05-5068136243a7 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:29.636575 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700de1b2-ef62-46bf-ab05-5068136243a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:29.636860 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700de1b2-ef62-46bf-ab05-5068136243a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:29.651027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700de1b2-ef62-46bf-ab05-5068136243a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-700de1b2-ef62-46bf-ab05-5068136243a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:38:29.658983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:29.660023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:38:29.660863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:29.666360 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b96f6-cc29-46a4-a08c-95b2a8c9b205 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:29.673480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 633/1208] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:38:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:29.716057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:29.830701 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b96f6-cc29-46a4-a08c-95b2a8c9b205 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:29.909512 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5cf14-9e62-498f-a1e9-998d67b41043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:29.909963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5cf14-9e62-498f-a1e9-998d67b41043 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:29.910213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5cf14-9e62-498f-a1e9-998d67b41043 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:29.910473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5cf14-9e62-498f-a1e9-998d67b41043 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:29.911450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5cf14-9e62-498f-a1e9-998d67b41043 None admin] RBAC: Adding query filter params (name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:29.912029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5cf14-9e62-498f-a1e9-998d67b41043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:29.913841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5cf14-9e62-498f-a1e9-998d67b41043 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:29.928976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 576/1209] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:38:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:38:29.939250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700de1b2-ef62-46bf-ab05-5068136243a7 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:30.007997 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:30.083997 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700de1b2-ef62-46bf-ab05-5068136243a7 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:30.134578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb682596-ae60-47ab-86f3-8834bd1b8999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:30.135272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb682596-ae60-47ab-86f3-8834bd1b8999 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:30.135559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb682596-ae60-47ab-86f3-8834bd1b8999 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:30.135839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb682596-ae60-47ab-86f3-8834bd1b8999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:30.149389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb682596-ae60-47ab-86f3-8834bd1b8999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:30.150917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb682596-ae60-47ab-86f3-8834bd1b8999 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-bb682596-ae60-47ab-86f3-8834bd1b8999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:30.173453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:30.175201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:30.176843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:30.176843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:30.176843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:30.176843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:30.176843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 634/1210] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:30.180828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5cf14-9e62-498f-a1e9-998d67b41043 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:30.393884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:30.402057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5cf14-9e62-498f-a1e9-998d67b41043 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:30.460230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3cd48-b87d-4459-9b13-bb328302dd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:30.461868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3cd48-b87d-4459-9b13-bb328302dd52 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:30.462355 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3cd48-b87d-4459-9b13-bb328302dd52 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:30.462849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3cd48-b87d-4459-9b13-bb328302dd52 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:30.464192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3cd48-b87d-4459-9b13-bb328302dd52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:30.464721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3cd48-b87d-4459-9b13-bb328302dd52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:30.466994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3cd48-b87d-4459-9b13-bb328302dd52 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:30.486793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 577/1211] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:38:30.493669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb682596-ae60-47ab-86f3-8834bd1b8999 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:30.634447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb682596-ae60-47ab-86f3-8834bd1b8999 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:30.636358 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:30.683200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32a24d-6946-4765-b030-109227f9b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:30.684033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32a24d-6946-4765-b030-109227f9b1fb None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:30.684320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32a24d-6946-4765-b030-109227f9b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:30.684595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32a24d-6946-4765-b030-109227f9b1fb None admin] PATH_INFO: `/v3/users/placement` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:30.693938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32a24d-6946-4765-b030-109227f9b1fb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:30.696185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32a24d-6946-4765-b030-109227f9b1fb None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-4c32a24d-6946-4765-b030-109227f9b1fb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:38:30.722713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:30.724670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:30.727229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:30.727229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:30.727229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:38:30.727229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:38:30.727229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:38:30.727229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:38:30.727229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:38:30.727229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:38:30.727229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:30.727229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:30.727229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 635/1212] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:30.730077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3cd48-b87d-4459-9b13-bb328302dd52 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:30.920604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3cd48-b87d-4459-9b13-bb328302dd52 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:30.993002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914a092-00cb-4c1f-9a50-052e19c7b9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:30.993002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914a092-00cb-4c1f-9a50-052e19c7b9a6 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:30.993441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914a092-00cb-4c1f-9a50-052e19c7b9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:30.993617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914a092-00cb-4c1f-9a50-052e19c7b9a6 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:30.995176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914a092-00cb-4c1f-9a50-052e19c7b9a6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:30.995684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914a092-00cb-4c1f-9a50-052e19c7b9a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:30.998183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914a092-00cb-4c1f-9a50-052e19c7b9a6 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:31.033409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 578/1213] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jul 19 20:38:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:38:31.040975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32a24d-6946-4765-b030-109227f9b1fb None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:31.201750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32a24d-6946-4765-b030-109227f9b1fb None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:31.244140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f3750-104e-45db-a4f3-87720239fd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:31.244818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3750-104e-45db-a4f3-87720239fd25 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:31.245260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3750-104e-45db-a4f3-87720239fd25 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:31.245479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3750-104e-45db-a4f3-87720239fd25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:31.284340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:31.348761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f3750-104e-45db-a4f3-87720239fd25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:31.350380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f3750-104e-45db-a4f3-87720239fd25 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:31.493535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 636/1214] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:38:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:38:31.500187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914a092-00cb-4c1f-9a50-052e19c7b9a6 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:31.524011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:31.660726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914a092-00cb-4c1f-9a50-052e19c7b9a6 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:31.726412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4dfb5-5fd7-4781-be61-3e1c4035f8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:31.728228 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4dfb5-5fd7-4781-be61-3e1c4035f8ae None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:31.728681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4dfb5-5fd7-4781-be61-3e1c4035f8ae None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:31.729121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4dfb5-5fd7-4781-be61-3e1c4035f8ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:31.747727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4dfb5-5fd7-4781-be61-3e1c4035f8ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:31.751972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4dfb5-5fd7-4781-be61-3e1c4035f8ae None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-58b4dfb5-5fd7-4781-be61-3e1c4035f8ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:31.784762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:31.786463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:31.792471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:31.792471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:31.792471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:31.792471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:31.792471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 579/1215] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:31.797202 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f3750-104e-45db-a4f3-87720239fd25 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:31.921901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:32.004223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f3750-104e-45db-a4f3-87720239fd25 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:32.082994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd05de9-69b5-4660-a1f8-f1c791c91101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:32.084164 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd05de9-69b5-4660-a1f8-f1c791c91101 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:32.084800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd05de9-69b5-4660-a1f8-f1c791c91101 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:32.085105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd05de9-69b5-4660-a1f8-f1c791c91101 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:32.087103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd05de9-69b5-4660-a1f8-f1c791c91101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:32.087614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd05de9-69b5-4660-a1f8-f1c791c91101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:32.090201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd05de9-69b5-4660-a1f8-f1c791c91101 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:32.120714 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 637/1216] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:32.123801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4dfb5-5fd7-4781-be61-3e1c4035f8ae None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:32.260559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:32.320940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4dfb5-5fd7-4781-be61-3e1c4035f8ae None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:32.387722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16888537-688e-45b0-aaad-58bf8bb5728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:32.389075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16888537-688e-45b0-aaad-58bf8bb5728a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:32.389536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16888537-688e-45b0-aaad-58bf8bb5728a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:32.390125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16888537-688e-45b0-aaad-58bf8bb5728a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:32.400425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16888537-688e-45b0-aaad-58bf8bb5728a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:32.403388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16888537-688e-45b0-aaad-58bf8bb5728a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-16888537-688e-45b0-aaad-58bf8bb5728a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:38:32.432571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:32.434787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:32.436567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:38:32.436567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:38:32.436567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:38:32.436567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:38:32.436567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:38:32.436567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:38:32.436567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:32.436567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:32.436567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 580/1217] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:32.446454 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd05de9-69b5-4660-a1f8-f1c791c91101 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:32.640519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd05de9-69b5-4660-a1f8-f1c791c91101 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:32.676133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:32.697483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3713ff7-920b-4607-9e71-bfd3e21618d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:32.698649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3713ff7-920b-4607-9e71-bfd3e21618d9 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:32.699163 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3713ff7-920b-4607-9e71-bfd3e21618d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:32.699658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3713ff7-920b-4607-9e71-bfd3e21618d9 None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:32.701190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3713ff7-920b-4607-9e71-bfd3e21618d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:32.701693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3713ff7-920b-4607-9e71-bfd3e21618d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:32.704650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3713ff7-920b-4607-9e71-bfd3e21618d9 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:32.727535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 638/1218] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:32.740655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16888537-688e-45b0-aaad-58bf8bb5728a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:32.931715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16888537-688e-45b0-aaad-58bf8bb5728a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:32.979597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:32.987646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb45a52-cf3e-4e1f-8815-b5c317615493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:32.988831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45a52-cf3e-4e1f-8815-b5c317615493 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:32.989574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45a52-cf3e-4e1f-8815-b5c317615493 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:32.990050 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45a52-cf3e-4e1f-8815-b5c317615493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:32.991360 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb45a52-cf3e-4e1f-8815-b5c317615493 None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=f27871b39b9b4edaa512bd5b7467174a) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:32.991831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb45a52-cf3e-4e1f-8815-b5c317615493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:32.995310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb45a52-cf3e-4e1f-8815-b5c317615493 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:33.005657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 581/1219] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:38:32 2022] GET /identity/v3/role_assignments?user.id=f27871b39b9b4edaa512bd5b7467174a&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:35.496617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35c7c-8ec8-4624-ae61-ac42f49fe170 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:35.497182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35c7c-8ec8-4624-ae61-ac42f49fe170 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:35.497621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35c7c-8ec8-4624-ae61-ac42f49fe170 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:35.499470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 639/1220] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:38:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:38:35.506212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904801d-aeca-49aa-98f2-8535d82b681f None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:35.507085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904801d-aeca-49aa-98f2-8535d82b681f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:35.507309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904801d-aeca-49aa-98f2-8535d82b681f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:35.529064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-5904801d-aeca-49aa-98f2-8535d82b681f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:38:35.530690 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5904801d-aeca-49aa-98f2-8535d82b681f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:38:35.612527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 582/1221] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:38:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:38:35.639674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc35c7c-8ec8-4624-ae61-ac42f49fe170 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:35.793295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc35c7c-8ec8-4624-ae61-ac42f49fe170 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:35.839184 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b120528-ed5e-435a-a817-23111bc64ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:35.840026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b120528-ed5e-435a-a817-23111bc64ba6 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:35.840317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b120528-ed5e-435a-a817-23111bc64ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:35.840598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b120528-ed5e-435a-a817-23111bc64ba6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:35.855966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b120528-ed5e-435a-a817-23111bc64ba6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-5b120528-ed5e-435a-a817-23111bc64ba6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:38:35.871415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:35.873425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:38:35.875269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:35.877573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 640/1222] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:35.883015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904801d-aeca-49aa-98f2-8535d82b681f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:36.045486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904801d-aeca-49aa-98f2-8535d82b681f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:36.078742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:36.091595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e261ec1-b1a7-4b59-bbc5-fb62b612f8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:36.092316 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e261ec1-b1a7-4b59-bbc5-fb62b612f8e6 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:36.092620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e261ec1-b1a7-4b59-bbc5-fb62b612f8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:36.092899 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e261ec1-b1a7-4b59-bbc5-fb62b612f8e6 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:36.093887 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e261ec1-b1a7-4b59-bbc5-fb62b612f8e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:36.094249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e261ec1-b1a7-4b59-bbc5-fb62b612f8e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:36.095757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e261ec1-b1a7-4b59-bbc5-fb62b612f8e6 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:36.105605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 583/1223] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:38:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:36.112844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b120528-ed5e-435a-a817-23111bc64ba6 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:36.184155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:36.264193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b120528-ed5e-435a-a817-23111bc64ba6 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:36.313749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ba35d-b8d3-4364-b5a6-a46ea8853926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:36.314599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ba35d-b8d3-4364-b5a6-a46ea8853926 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:36.314861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ba35d-b8d3-4364-b5a6-a46ea8853926 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:36.315206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ba35d-b8d3-4364-b5a6-a46ea8853926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:36.331177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ba35d-b8d3-4364-b5a6-a46ea8853926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:36.332857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ba35d-b8d3-4364-b5a6-a46ea8853926 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-c11ba35d-b8d3-4364-b5a6-a46ea8853926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:36.355242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:36.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:36.359315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:36.359315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:36.359315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:36.359315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:36.359315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 641/1224] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:38:36.363399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e261ec1-b1a7-4b59-bbc5-fb62b612f8e6 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:36.516423 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e261ec1-b1a7-4b59-bbc5-fb62b612f8e6 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:36.556339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f88cfd-1329-408f-8a88-dbfd230131e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:36.557013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f88cfd-1329-408f-8a88-dbfd230131e0 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:36.557299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f88cfd-1329-408f-8a88-dbfd230131e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:36.557608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f88cfd-1329-408f-8a88-dbfd230131e0 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:36.558369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f88cfd-1329-408f-8a88-dbfd230131e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:36.558676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f88cfd-1329-408f-8a88-dbfd230131e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:36.560070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f88cfd-1329-408f-8a88-dbfd230131e0 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:36.574553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 584/1225] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:38:36.581140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ba35d-b8d3-4364-b5a6-a46ea8853926 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:36.710374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:36.720188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ba35d-b8d3-4364-b5a6-a46ea8853926 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:36.762243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e281f7-efa6-4906-909a-7bf5f532d00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:36.763117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e281f7-efa6-4906-909a-7bf5f532d00c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:36.763497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e281f7-efa6-4906-909a-7bf5f532d00c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:36.763792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e281f7-efa6-4906-909a-7bf5f532d00c None admin] PATH_INFO: `/v3/users/placement` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:36.775253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e281f7-efa6-4906-909a-7bf5f532d00c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:36.776733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e281f7-efa6-4906-909a-7bf5f532d00c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-53e281f7-efa6-4906-909a-7bf5f532d00c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:38:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:36.795890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:36.797889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:36.797889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:36.797889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:38:36.797889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:38:36.797889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:38:36.797889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:38:36.797889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:38:36.797889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:38:36.797889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:36.797889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:36.800376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 642/1226] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:36.801687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f88cfd-1329-408f-8a88-dbfd230131e0 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:36.875669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:36.955309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f88cfd-1329-408f-8a88-dbfd230131e0 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:36.996049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ce30d-8eec-4a37-a2dc-60cc1ff8fba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:36.996736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ce30d-8eec-4a37-a2dc-60cc1ff8fba9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:36.997016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ce30d-8eec-4a37-a2dc-60cc1ff8fba9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:36.997399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ce30d-8eec-4a37-a2dc-60cc1ff8fba9 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:36.998540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ce30d-8eec-4a37-a2dc-60cc1ff8fba9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:36.998854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ce30d-8eec-4a37-a2dc-60cc1ff8fba9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:37.000309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ce30d-8eec-4a37-a2dc-60cc1ff8fba9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:37.026479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 585/1227] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jul 19 20:38:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:37.035020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e281f7-efa6-4906-909a-7bf5f532d00c None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:37.162481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e281f7-efa6-4906-909a-7bf5f532d00c None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:37.209376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401a5b-6cb7-4c0a-96da-9970234d3a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:37.210208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401a5b-6cb7-4c0a-96da-9970234d3a19 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:37.210491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401a5b-6cb7-4c0a-96da-9970234d3a19 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:37.210762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401a5b-6cb7-4c0a-96da-9970234d3a19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:37.223869 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca401a5b-6cb7-4c0a-96da-9970234d3a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:37.226675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca401a5b-6cb7-4c0a-96da-9970234d3a19 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-ca401a5b-6cb7-4c0a-96da-9970234d3a19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:37.252936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:37.254684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:37.261952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:37.261952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:37.261952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:37.261952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:37.261952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 643/1228] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:37.262865 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ce30d-8eec-4a37-a2dc-60cc1ff8fba9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:37.396122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ce30d-8eec-4a37-a2dc-60cc1ff8fba9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:37.398635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:37.434384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2265c93-1e30-4187-a320-4e9124b4d800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:37.434692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2265c93-1e30-4187-a320-4e9124b4d800 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:37.434991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2265c93-1e30-4187-a320-4e9124b4d800 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:37.435286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2265c93-1e30-4187-a320-4e9124b4d800 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:37.436022 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2265c93-1e30-4187-a320-4e9124b4d800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:37.436337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2265c93-1e30-4187-a320-4e9124b4d800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:37.437826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2265c93-1e30-4187-a320-4e9124b4d800 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:37.450677 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 586/1229] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:37.457584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401a5b-6cb7-4c0a-96da-9970234d3a19 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:37.467999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:37.576237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401a5b-6cb7-4c0a-96da-9970234d3a19 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:37.619475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9fe32-6834-401d-a17a-3edd801b2a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:37.620886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9fe32-6834-401d-a17a-3edd801b2a44 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:37.621432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9fe32-6834-401d-a17a-3edd801b2a44 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:37.622053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9fe32-6834-401d-a17a-3edd801b2a44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:37.632697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9fe32-6834-401d-a17a-3edd801b2a44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:37.634223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9fe32-6834-401d-a17a-3edd801b2a44 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-6ba9fe32-6834-401d-a17a-3edd801b2a44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:38:37.655987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:37.657264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:37.659041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:38:37.659041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:38:37.659041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:38:37.659041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:38:37.659041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:38:37.659041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:38:37.659041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:37.659041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:37.659041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 644/1230] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:38:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:37.663273 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2265c93-1e30-4187-a320-4e9124b4d800 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:37.784857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2265c93-1e30-4187-a320-4e9124b4d800 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:37.831423 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1e968-9c38-401d-bed7-aae4c6fd4b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:37.835074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1e968-9c38-401d-bed7-aae4c6fd4b8b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:37.835386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1e968-9c38-401d-bed7-aae4c6fd4b8b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:37.835656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1e968-9c38-401d-bed7-aae4c6fd4b8b None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:37.836984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1e968-9c38-401d-bed7-aae4c6fd4b8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:37.837343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1e968-9c38-401d-bed7-aae4c6fd4b8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:37.838842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1e968-9c38-401d-bed7-aae4c6fd4b8b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:37.856337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 587/1231] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:38:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:37.865791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9fe32-6834-401d-a17a-3edd801b2a44 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:37.969283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:38.021756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9fe32-6834-401d-a17a-3edd801b2a44 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:38.069949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c2812-e616-4485-8369-3dbfa9827cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:38.071639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c2812-e616-4485-8369-3dbfa9827cc2 None admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:38.071940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c2812-e616-4485-8369-3dbfa9827cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:38.072232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c2812-e616-4485-8369-3dbfa9827cc2 None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/f27871b39b9b4edaa512bd5b7467174a/roles/2de43a7c794c4df181f6f4c58a83a9bf` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:38.092036 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c2812-e616-4485-8369-3dbfa9827cc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=f27871b39b9b4edaa512bd5b7467174a, role_id=2de43a7c794c4df181f6f4c58a83a9bf)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:38.093535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c2812-e616-4485-8369-3dbfa9827cc2 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:38.110404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 645/1232] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:38:37 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/f27871b39b9b4edaa512bd5b7467174a/roles/2de43a7c794c4df181f6f4c58a83a9bf => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:38:40.494808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba9662-82ed-4717-9ab4-79ec132897e7 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:40.495748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba9662-82ed-4717-9ab4-79ec132897e7 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:40.495748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba9662-82ed-4717-9ab4-79ec132897e7 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:40.497909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 588/1233] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:38:40.503869 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a783a-a009-4b63-b87e-08e1d4530cb8 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:40.504432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a783a-a009-4b63-b87e-08e1d4530cb8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:40.504432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a783a-a009-4b63-b87e-08e1d4530cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:40.535151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-801a783a-a009-4b63-b87e-08e1d4530cb8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:38:40.536385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-801a783a-a009-4b63-b87e-08e1d4530cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:38:40.622026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 646/1234] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:38:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:38:40.649543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba9662-82ed-4717-9ab4-79ec132897e7 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:40.683366 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:40.790001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba9662-82ed-4717-9ab4-79ec132897e7 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:40.836725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d983d08-a9ae-41e1-9f4b-b61d2989b059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:40.837718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d983d08-a9ae-41e1-9f4b-b61d2989b059 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:40.838008 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d983d08-a9ae-41e1-9f4b-b61d2989b059 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:40.838353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d983d08-a9ae-41e1-9f4b-b61d2989b059 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:40.850922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d983d08-a9ae-41e1-9f4b-b61d2989b059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-0d983d08-a9ae-41e1-9f4b-b61d2989b059 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:38:40.859020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:40.861224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:40.863573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 589/1235] 10.222.0.70 () {56 vars in 1102 bytes} [Tue Jul 19 20:38:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:40.866988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a783a-a009-4b63-b87e-08e1d4530cb8 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:40.948934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:41.031561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a783a-a009-4b63-b87e-08e1d4530cb8 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:41.077043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfcaa7-a86c-42e9-a671-5fadfb90a5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:41.078158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfcaa7-a86c-42e9-a671-5fadfb90a5b6 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:41.078604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfcaa7-a86c-42e9-a671-5fadfb90a5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:41.078926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfcaa7-a86c-42e9-a671-5fadfb90a5b6 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:41.079869 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfcaa7-a86c-42e9-a671-5fadfb90a5b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:41.080190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfcaa7-a86c-42e9-a671-5fadfb90a5b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:41.081808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfcaa7-a86c-42e9-a671-5fadfb90a5b6 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:41.092783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 647/1236] 10.222.0.70 () {56 vars in 1103 bytes} [Tue Jul 19 20:38:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:38:41.101400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d983d08-a9ae-41e1-9f4b-b61d2989b059 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:41.217256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:41.245625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d983d08-a9ae-41e1-9f4b-b61d2989b059 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:41.287708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb297db7-076c-4acf-b606-84b5f3894dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:41.288546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb297db7-076c-4acf-b606-84b5f3894dff None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:41.289457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb297db7-076c-4acf-b606-84b5f3894dff None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:41.289795 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb297db7-076c-4acf-b606-84b5f3894dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:41.305544 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb297db7-076c-4acf-b606-84b5f3894dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:41.307247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb297db7-076c-4acf-b606-84b5f3894dff None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-bb297db7-076c-4acf-b606-84b5f3894dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:41.326597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:41.328849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:41.330702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:41.330702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:41.330702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:41.330702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:41.330702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 590/1237] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:41.334291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfcaa7-a86c-42e9-a671-5fadfb90a5b6 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:41.483533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfcaa7-a86c-42e9-a671-5fadfb90a5b6 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:41.499586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:41.541394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dee0c6-2096-4d3f-9567-6003079b98da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:41.542375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee0c6-2096-4d3f-9567-6003079b98da None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:41.542859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee0c6-2096-4d3f-9567-6003079b98da None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:41.543218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee0c6-2096-4d3f-9567-6003079b98da None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:41.544122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dee0c6-2096-4d3f-9567-6003079b98da None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:41.544558 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dee0c6-2096-4d3f-9567-6003079b98da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:41.547166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dee0c6-2096-4d3f-9567-6003079b98da None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:41.565843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 648/1238] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:41.575522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb297db7-076c-4acf-b606-84b5f3894dff None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:41.744254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb297db7-076c-4acf-b606-84b5f3894dff None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:41.795694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd068e-b00d-4e92-ad56-dda08e250be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:41.797293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd068e-b00d-4e92-ad56-dda08e250be9 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:41.797875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd068e-b00d-4e92-ad56-dda08e250be9 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:41.798372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd068e-b00d-4e92-ad56-dda08e250be9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:41.804078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:41.812696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd068e-b00d-4e92-ad56-dda08e250be9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:41.814493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd068e-b00d-4e92-ad56-dda08e250be9 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-c7fd068e-b00d-4e92-ad56-dda08e250be9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:38:41.832170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:41.834183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:41.835721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:41.835721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:41.835721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:38:41.835721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:38:41.835721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:38:41.835721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:38:41.835721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:38:41.835721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:38:41.835721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:41.835721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:41.835721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 591/1239] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:41.850430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dee0c6-2096-4d3f-9567-6003079b98da None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:42.009583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dee0c6-2096-4d3f-9567-6003079b98da None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:42.047777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d47540-6fd5-4c36-8159-d4c2a0beeb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:42.048465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d47540-6fd5-4c36-8159-d4c2a0beeb6b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:42.048923 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d47540-6fd5-4c36-8159-d4c2a0beeb6b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:42.049106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d47540-6fd5-4c36-8159-d4c2a0beeb6b None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:42.049973 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d47540-6fd5-4c36-8159-d4c2a0beeb6b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:42.050310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d47540-6fd5-4c36-8159-d4c2a0beeb6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:42.051836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d47540-6fd5-4c36-8159-d4c2a0beeb6b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:42.079783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 649/1240] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jul 19 20:38:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:38:42.086943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd068e-b00d-4e92-ad56-dda08e250be9 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:42.221452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd068e-b00d-4e92-ad56-dda08e250be9 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:42.275093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68207dcb-fcad-46b8-b28d-d4ad9d3f2b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:42.275678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68207dcb-fcad-46b8-b28d-d4ad9d3f2b83 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:42.275969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68207dcb-fcad-46b8-b28d-d4ad9d3f2b83 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:42.276252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68207dcb-fcad-46b8-b28d-d4ad9d3f2b83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:42.365678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:42.381650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68207dcb-fcad-46b8-b28d-d4ad9d3f2b83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:42.385846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68207dcb-fcad-46b8-b28d-d4ad9d3f2b83 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:42.535860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 592/1241] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:38:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 20:38:42.543092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d47540-6fd5-4c36-8159-d4c2a0beeb6b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:42.580801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:42.683231 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d47540-6fd5-4c36-8159-d4c2a0beeb6b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:42.727757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f905-c24a-41a8-b9c9-e3f546556982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:42.728978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f905-c24a-41a8-b9c9-e3f546556982 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:42.729334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f905-c24a-41a8-b9c9-e3f546556982 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:42.729719 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f905-c24a-41a8-b9c9-e3f546556982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:42.743419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2f905-c24a-41a8-b9c9-e3f546556982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:42.745744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2f905-c24a-41a8-b9c9-e3f546556982 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-1ef2f905-c24a-41a8-b9c9-e3f546556982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:42.766552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:42.768683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:42.770746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:42.770746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:42.770746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:42.770746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:42.773007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 650/1242] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:42.774194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68207dcb-fcad-46b8-b28d-d4ad9d3f2b83 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:42.893995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:42.901883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68207dcb-fcad-46b8-b28d-d4ad9d3f2b83 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:42.953959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1aae9-a868-4ad8-8c9d-6af87eb3d2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:42.954883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1aae9-a868-4ad8-8c9d-6af87eb3d2db None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:42.955334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1aae9-a868-4ad8-8c9d-6af87eb3d2db None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:42.955952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1aae9-a868-4ad8-8c9d-6af87eb3d2db None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:42.957168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1aae9-a868-4ad8-8c9d-6af87eb3d2db None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:42.957634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1aae9-a868-4ad8-8c9d-6af87eb3d2db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:42.959446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1aae9-a868-4ad8-8c9d-6af87eb3d2db None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:42.975783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 593/1243] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:38:42.981943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f905-c24a-41a8-b9c9-e3f546556982 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:43.082866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:43.112633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f905-c24a-41a8-b9c9-e3f546556982 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:43.161929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb50d2-956b-4c57-86af-89a0ecaeef69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:43.162910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb50d2-956b-4c57-86af-89a0ecaeef69 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:43.163388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb50d2-956b-4c57-86af-89a0ecaeef69 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:43.163745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb50d2-956b-4c57-86af-89a0ecaeef69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:43.172107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb50d2-956b-4c57-86af-89a0ecaeef69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:43.173838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb50d2-956b-4c57-86af-89a0ecaeef69 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-27bb50d2-956b-4c57-86af-89a0ecaeef69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:38:43.190451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:43.192532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:43.194588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:38:43.194588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:38:43.194588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:38:43.194588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:38:43.194588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:38:43.194588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:38:43.194588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:43.194588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:43.194588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 651/1244] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:38:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:38:43.205816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1aae9-a868-4ad8-8c9d-6af87eb3d2db None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:43.362104 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1aae9-a868-4ad8-8c9d-6af87eb3d2db None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:43.412821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e0436-3233-47a0-bc78-55c7927c4f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:43.413474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e0436-3233-47a0-bc78-55c7927c4f02 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:43.413767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e0436-3233-47a0-bc78-55c7927c4f02 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:43.414558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e0436-3233-47a0-bc78-55c7927c4f02 None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:43.415447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e0436-3233-47a0-bc78-55c7927c4f02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:43.415759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e0436-3233-47a0-bc78-55c7927c4f02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:43.417432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e0436-3233-47a0-bc78-55c7927c4f02 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:43.431902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 594/1245] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:38:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:43.438456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb50d2-956b-4c57-86af-89a0ecaeef69 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:43.602131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb50d2-956b-4c57-86af-89a0ecaeef69 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:43.646432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38068b8c-2a5f-46f9-ac69-a26cc777854c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:43.647165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38068b8c-2a5f-46f9-ac69-a26cc777854c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:43.647492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38068b8c-2a5f-46f9-ac69-a26cc777854c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:43.647810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38068b8c-2a5f-46f9-ac69-a26cc777854c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:43.649424 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38068b8c-2a5f-46f9-ac69-a26cc777854c None admin] RBAC: Adding query filter params (role.id=2de43a7c794c4df181f6f4c58a83a9bf, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=f27871b39b9b4edaa512bd5b7467174a) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:43.649757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38068b8c-2a5f-46f9-ac69-a26cc777854c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:43.651294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38068b8c-2a5f-46f9-ac69-a26cc777854c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:43.658624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 652/1246] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:38:43 2022] GET /identity/v3/role_assignments?user.id=f27871b39b9b4edaa512bd5b7467174a&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=2de43a7c794c4df181f6f4c58a83a9bf => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:46.162769 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc217d8e-cf6b-418d-b2e9-65419d45e55f None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:46.163634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc217d8e-cf6b-418d-b2e9-65419d45e55f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:46.163634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc217d8e-cf6b-418d-b2e9-65419d45e55f None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:46.166049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 595/1247] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:38:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:38:46.172517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94435cbd-d88d-4027-89c3-80c93ab2742d None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:46.173323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94435cbd-d88d-4027-89c3-80c93ab2742d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:46.173323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94435cbd-d88d-4027-89c3-80c93ab2742d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:46.179914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:46.196008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-94435cbd-d88d-4027-89c3-80c93ab2742d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:38:46.197283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-94435cbd-d88d-4027-89c3-80c93ab2742d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:38:46.278874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 653/1248] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:38:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:38:46.305742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc217d8e-cf6b-418d-b2e9-65419d45e55f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:46.328307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:46.434621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc217d8e-cf6b-418d-b2e9-65419d45e55f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:46.481264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8e41e-8128-465d-a882-973b4f2f40e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:46.482317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8e41e-8128-465d-a882-973b4f2f40e1 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:46.482755 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8e41e-8128-465d-a882-973b4f2f40e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:46.483116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8e41e-8128-465d-a882-973b4f2f40e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:46.497106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8e41e-8128-465d-a882-973b4f2f40e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-98d8e41e-8128-465d-a882-973b4f2f40e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:38:46.505729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:46.507293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:46.508852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 596/1249] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:38:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:38:46.514267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94435cbd-d88d-4027-89c3-80c93ab2742d None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:46.640650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94435cbd-d88d-4027-89c3-80c93ab2742d None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:46.686135 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:46.702178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cb54b-5a58-4a79-b150-ce036fa1c4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:46.702178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cb54b-5a58-4a79-b150-ce036fa1c4e5 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:46.702178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cb54b-5a58-4a79-b150-ce036fa1c4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:46.702178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cb54b-5a58-4a79-b150-ce036fa1c4e5 None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:46.702891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cb54b-5a58-4a79-b150-ce036fa1c4e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:46.702891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cb54b-5a58-4a79-b150-ce036fa1c4e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:46.702891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cb54b-5a58-4a79-b150-ce036fa1c4e5 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:46.711378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 654/1250] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:38:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:46.717757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8e41e-8128-465d-a882-973b4f2f40e1 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:46.850569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:46.887836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8e41e-8128-465d-a882-973b4f2f40e1 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:46.954745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279179a7-cb04-49b5-b72e-82d1d905eeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:46.956188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279179a7-cb04-49b5-b72e-82d1d905eeb3 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:46.956818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279179a7-cb04-49b5-b72e-82d1d905eeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:46.957550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279179a7-cb04-49b5-b72e-82d1d905eeb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:46.980180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279179a7-cb04-49b5-b72e-82d1d905eeb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:46.982852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279179a7-cb04-49b5-b72e-82d1d905eeb3 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-279179a7-cb04-49b5-b72e-82d1d905eeb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:47.017511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:47.019465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:47.021212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:47.021212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:47.021212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:47.021212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:47.021212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 597/1251] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:47.028956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cb54b-5a58-4a79-b150-ce036fa1c4e5 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:47.219674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cb54b-5a58-4a79-b150-ce036fa1c4e5 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:47.298632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b2b51-3ef6-4fdd-a188-efb3d9ff64b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:47.299444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b2b51-3ef6-4fdd-a188-efb3d9ff64b9 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:47.299920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b2b51-3ef6-4fdd-a188-efb3d9ff64b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:47.300457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b2b51-3ef6-4fdd-a188-efb3d9ff64b9 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:47.301293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b2b51-3ef6-4fdd-a188-efb3d9ff64b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:47.301725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b2b51-3ef6-4fdd-a188-efb3d9ff64b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:47.303313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b2b51-3ef6-4fdd-a188-efb3d9ff64b9 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:47.325154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 655/1252] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:47.332890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279179a7-cb04-49b5-b72e-82d1d905eeb3 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:47.512897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279179a7-cb04-49b5-b72e-82d1d905eeb3 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:47.555891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:47.567068 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3dda4-1780-4acc-998a-fa871ce98952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:47.568102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3dda4-1780-4acc-998a-fa871ce98952 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:47.568592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3dda4-1780-4acc-998a-fa871ce98952 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:47.569078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3dda4-1780-4acc-998a-fa871ce98952 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:47.582466 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3dda4-1780-4acc-998a-fa871ce98952 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:47.584336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3dda4-1780-4acc-998a-fa871ce98952 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-a1f3dda4-1780-4acc-998a-fa871ce98952 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:38:47.606431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:47.607903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:47.609178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:47.609178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:47.609178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:38:47.609178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:38:47.609178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:38:47.609178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:38:47.609178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:38:47.609178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:38:47.609178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:47.609178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:47.611709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 598/1253] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:38:47.618958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b2b51-3ef6-4fdd-a188-efb3d9ff64b9 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:47.703669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:47.780584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b2b51-3ef6-4fdd-a188-efb3d9ff64b9 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:47.833140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b971a-ebad-40f0-afc2-17604b302a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:47.833953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b971a-ebad-40f0-afc2-17604b302a92 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:47.834299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b971a-ebad-40f0-afc2-17604b302a92 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:47.834616 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b971a-ebad-40f0-afc2-17604b302a92 None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:47.835499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b971a-ebad-40f0-afc2-17604b302a92 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:47.835859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b971a-ebad-40f0-afc2-17604b302a92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:47.837826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b971a-ebad-40f0-afc2-17604b302a92 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:47.865338 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 656/1254] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jul 19 20:38:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:47.871911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3dda4-1780-4acc-998a-fa871ce98952 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:48.026439 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3dda4-1780-4acc-998a-fa871ce98952 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:48.071576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe6e9e-2711-4a78-bb16-651cdf17362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:48.072300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe6e9e-2711-4a78-bb16-651cdf17362c None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:48.072578 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe6e9e-2711-4a78-bb16-651cdf17362c None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:48.072850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe6e9e-2711-4a78-bb16-651cdf17362c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:48.154523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:48.173946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe6e9e-2711-4a78-bb16-651cdf17362c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:48.175481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe6e9e-2711-4a78-bb16-651cdf17362c None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:48.321391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 599/1255] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:38:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:38:48.329577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b971a-ebad-40f0-afc2-17604b302a92 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:48.466126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b971a-ebad-40f0-afc2-17604b302a92 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:48.484264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:48.513401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fd48c-3286-4da9-ba90-34d81e2cabe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:48.514300 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fd48c-3286-4da9-ba90-34d81e2cabe9 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:48.514566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fd48c-3286-4da9-ba90-34d81e2cabe9 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:48.514990 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fd48c-3286-4da9-ba90-34d81e2cabe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:48.529471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81fd48c-3286-4da9-ba90-34d81e2cabe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:48.531256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81fd48c-3286-4da9-ba90-34d81e2cabe9 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-b81fd48c-3286-4da9-ba90-34d81e2cabe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:48.554087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:48.556009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:48.557623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:48.557623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:48.557623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:48.557623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:48.557623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 657/1256] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:38:48.563676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe6e9e-2711-4a78-bb16-651cdf17362c None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:48.686389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:48.700646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe6e9e-2711-4a78-bb16-651cdf17362c None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:48.760611 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f537c8-c72f-4101-81c5-a721f172df26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:48.761292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f537c8-c72f-4101-81c5-a721f172df26 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:48.761612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f537c8-c72f-4101-81c5-a721f172df26 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:48.761895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f537c8-c72f-4101-81c5-a721f172df26 None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:48.762632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f537c8-c72f-4101-81c5-a721f172df26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:48.762943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f537c8-c72f-4101-81c5-a721f172df26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:48.764476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f537c8-c72f-4101-81c5-a721f172df26 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:48.779282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 600/1257] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:48.785472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fd48c-3286-4da9-ba90-34d81e2cabe9 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:48.928647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fd48c-3286-4da9-ba90-34d81e2cabe9 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:48.984632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15357e4-c4c7-40ea-9ee0-1fe5dfed4832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:48.985850 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15357e4-c4c7-40ea-9ee0-1fe5dfed4832 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:48.987610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15357e4-c4c7-40ea-9ee0-1fe5dfed4832 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:48.987610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15357e4-c4c7-40ea-9ee0-1fe5dfed4832 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:48.996214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15357e4-c4c7-40ea-9ee0-1fe5dfed4832 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:48.997762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15357e4-c4c7-40ea-9ee0-1fe5dfed4832 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-e15357e4-c4c7-40ea-9ee0-1fe5dfed4832 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:38:49.012713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:49.014772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:49.016870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:38:49.016870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:38:49.016870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:38:49.016870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:38:49.016870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:38:49.016870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:38:49.016870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:49.016870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:49.016870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 658/1258] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:38:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:38:49.020908 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f537c8-c72f-4101-81c5-a721f172df26 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:49.154919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f537c8-c72f-4101-81c5-a721f172df26 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:49.200265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8725a8-2dca-4ae1-9f55-f7e1b716cf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:49.201002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8725a8-2dca-4ae1-9f55-f7e1b716cf3b None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:49.201335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8725a8-2dca-4ae1-9f55-f7e1b716cf3b None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:49.201776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8725a8-2dca-4ae1-9f55-f7e1b716cf3b None admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:49.202946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8725a8-2dca-4ae1-9f55-f7e1b716cf3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:49.203325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8725a8-2dca-4ae1-9f55-f7e1b716cf3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:49.204949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8725a8-2dca-4ae1-9f55-f7e1b716cf3b None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:49.221698 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 601/1259] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:38:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:38:49.230872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15357e4-c4c7-40ea-9ee0-1fe5dfed4832 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:49.268311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:49.403159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15357e4-c4c7-40ea-9ee0-1fe5dfed4832 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:49.447999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99126f-def1-46d2-9ca3-0de5188d2f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:49.448659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99126f-def1-46d2-9ca3-0de5188d2f27 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:49.448937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99126f-def1-46d2-9ca3-0de5188d2f27 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:49.449210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99126f-def1-46d2-9ca3-0de5188d2f27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:49.450106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99126f-def1-46d2-9ca3-0de5188d2f27 None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=f27871b39b9b4edaa512bd5b7467174a) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:49.450415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99126f-def1-46d2-9ca3-0de5188d2f27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:49.452492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99126f-def1-46d2-9ca3-0de5188d2f27 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:49.458712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 659/1260] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:38:49 2022] GET /identity/v3/role_assignments?user.id=f27871b39b9b4edaa512bd5b7467174a&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=85420c68300642aabfd730fb8fef8347 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:51.681987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1fa30-5ff6-4d79-949d-854e872602e2 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:51.682917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1fa30-5ff6-4d79-949d-854e872602e2 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:51.682917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1fa30-5ff6-4d79-949d-854e872602e2 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:51.683869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 602/1261] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:38:51.692180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1140c1-27a3-47ab-85af-33247ec68b28 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:51.692929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1140c1-27a3-47ab-85af-33247ec68b28 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:51.692929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1140c1-27a3-47ab-85af-33247ec68b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:51.717061 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-2e1140c1-27a3-47ab-85af-33247ec68b28 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:38:51.718894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2e1140c1-27a3-47ab-85af-33247ec68b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:38:51.830866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 660/1262] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:38:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:38:51.868345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1fa30-5ff6-4d79-949d-854e872602e2 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:51.885582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:52.036550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1fa30-5ff6-4d79-949d-854e872602e2 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:52.091055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b814159-ab26-416e-ab82-5415dbcf82fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:52.091902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b814159-ab26-416e-ab82-5415dbcf82fb None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:52.092185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b814159-ab26-416e-ab82-5415dbcf82fb None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:52.092458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b814159-ab26-416e-ab82-5415dbcf82fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:52.107425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b814159-ab26-416e-ab82-5415dbcf82fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-2b814159-ab26-416e-ab82-5415dbcf82fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:38:52.116455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:52.118399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:52.120041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 603/1263] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:38:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:52.125228 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1140c1-27a3-47ab-85af-33247ec68b28 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:52.152372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:52.307668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1140c1-27a3-47ab-85af-33247ec68b28 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:52.375384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017c0f5-2b61-44c8-925e-43681287ef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:52.375384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017c0f5-2b61-44c8-925e-43681287ef7d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:52.375384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017c0f5-2b61-44c8-925e-43681287ef7d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:52.375384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017c0f5-2b61-44c8-925e-43681287ef7d None admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:52.376913 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017c0f5-2b61-44c8-925e-43681287ef7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:52.376913 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017c0f5-2b61-44c8-925e-43681287ef7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:52.378002 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017c0f5-2b61-44c8-925e-43681287ef7d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:52.390888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 661/1264] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:38:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:52.398445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b814159-ab26-416e-ab82-5415dbcf82fb None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:52.487190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:52.539978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b814159-ab26-416e-ab82-5415dbcf82fb None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:52.585035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52df26-d987-44da-baa2-8db7e2d9cb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:52.586556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52df26-d987-44da-baa2-8db7e2d9cb7a None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:52.587059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52df26-d987-44da-baa2-8db7e2d9cb7a None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:52.588095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52df26-d987-44da-baa2-8db7e2d9cb7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:52.603068 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52df26-d987-44da-baa2-8db7e2d9cb7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:52.604909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52df26-d987-44da-baa2-8db7e2d9cb7a None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-ea52df26-d987-44da-baa2-8db7e2d9cb7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:52.628993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:52.631187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:52.633740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:52.633740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:52.633740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:52.633740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:52.633740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 604/1265] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:52.639640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017c0f5-2b61-44c8-925e-43681287ef7d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:52.772732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:52.799738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017c0f5-2b61-44c8-925e-43681287ef7d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:52.846158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a9bfa-8023-4fec-810a-558f5a5d7b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:52.846737 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a9bfa-8023-4fec-810a-558f5a5d7b3a None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:52.847105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a9bfa-8023-4fec-810a-558f5a5d7b3a None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:52.847460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a9bfa-8023-4fec-810a-558f5a5d7b3a None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:52.848225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a9bfa-8023-4fec-810a-558f5a5d7b3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:52.848534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a9bfa-8023-4fec-810a-558f5a5d7b3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:52.849966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a9bfa-8023-4fec-810a-558f5a5d7b3a None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:52.867791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 662/1266] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:52.875079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52df26-d987-44da-baa2-8db7e2d9cb7a None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:53.054405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52df26-d987-44da-baa2-8db7e2d9cb7a None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:53.089787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:53.107819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08091e65-40fa-45e1-ae8e-4e1fd51cf21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:53.108555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08091e65-40fa-45e1-ae8e-4e1fd51cf21d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:53.108842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08091e65-40fa-45e1-ae8e-4e1fd51cf21d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:53.109118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08091e65-40fa-45e1-ae8e-4e1fd51cf21d None admin] PATH_INFO: `/v3/users/placement` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:53.119364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08091e65-40fa-45e1-ae8e-4e1fd51cf21d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:53.121267 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08091e65-40fa-45e1-ae8e-4e1fd51cf21d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-08091e65-40fa-45e1-ae8e-4e1fd51cf21d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:38:53.137828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:53.139222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:53.140470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:53.140470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:53.140470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:38:53.140470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:38:53.140470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:38:53.140470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:38:53.140470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:38:53.140470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:38:53.140470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:53.140470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:53.140470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 605/1267] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:53.146603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a9bfa-8023-4fec-810a-558f5a5d7b3a None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:53.256436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a9bfa-8023-4fec-810a-558f5a5d7b3a None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:53.294274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22eab7-260d-4e7f-a387-7cb64aee7b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:53.294942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22eab7-260d-4e7f-a387-7cb64aee7b6e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:53.295354 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22eab7-260d-4e7f-a387-7cb64aee7b6e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:53.295626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22eab7-260d-4e7f-a387-7cb64aee7b6e None admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:53.296470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22eab7-260d-4e7f-a387-7cb64aee7b6e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:53.296771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22eab7-260d-4e7f-a387-7cb64aee7b6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:53.298682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22eab7-260d-4e7f-a387-7cb64aee7b6e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:53.309075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:53.320996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 663/1268] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jul 19 20:38:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:38:53.327202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08091e65-40fa-45e1-ae8e-4e1fd51cf21d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:53.448610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08091e65-40fa-45e1-ae8e-4e1fd51cf21d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:53.487044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2b2b6-5e41-4ba5-b0d8-f799b1f613f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:53.487843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2b2b6-5e41-4ba5-b0d8-f799b1f613f8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:53.488130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2b2b6-5e41-4ba5-b0d8-f799b1f613f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:53.488408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2b2b6-5e41-4ba5-b0d8-f799b1f613f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:53.500296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2b2b6-5e41-4ba5-b0d8-f799b1f613f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:53.501818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2b2b6-5e41-4ba5-b0d8-f799b1f613f8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-53b2b2b6-5e41-4ba5-b0d8-f799b1f613f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:53.519907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:53.521788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:53.524539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:53.524539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:53.524539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:53.524539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:53.524539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 606/1269] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:38:53.528846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22eab7-260d-4e7f-a387-7cb64aee7b6e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:53.634212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22eab7-260d-4e7f-a387-7cb64aee7b6e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:53.670460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d6d7-1b61-4ba9-98b7-90664b292eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:53.671123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d6d7-1b61-4ba9-98b7-90664b292eee None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:53.671405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d6d7-1b61-4ba9-98b7-90664b292eee None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:53.671685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946d6d7-1b61-4ba9-98b7-90664b292eee None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:53.672407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946d6d7-1b61-4ba9-98b7-90664b292eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:53.672713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946d6d7-1b61-4ba9-98b7-90664b292eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:53.674104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946d6d7-1b61-4ba9-98b7-90664b292eee None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:53.686636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 664/1270] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:53.695796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2b2b6-5e41-4ba5-b0d8-f799b1f613f8 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:53.729955 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:53.803774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2b2b6-5e41-4ba5-b0d8-f799b1f613f8 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:53.838599 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daff503-950d-4cf0-b38d-b1f9459d9bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:53.839388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daff503-950d-4cf0-b38d-b1f9459d9bdd None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:53.839666 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daff503-950d-4cf0-b38d-b1f9459d9bdd None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:53.839935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daff503-950d-4cf0-b38d-b1f9459d9bdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:53.846881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daff503-950d-4cf0-b38d-b1f9459d9bdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:53.848335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daff503-950d-4cf0-b38d-b1f9459d9bdd None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-3daff503-950d-4cf0-b38d-b1f9459d9bdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:38:53.864184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:53.865880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:53.869134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:38:53.869134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:38:53.869134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:38:53.869134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:38:53.869134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:38:53.869134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:38:53.869134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:53.869134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:53.869134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 607/1271] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:38:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:53.876494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d6d7-1b61-4ba9-98b7-90664b292eee None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:53.953614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:54.006499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946d6d7-1b61-4ba9-98b7-90664b292eee None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:54.048022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076caea7-49eb-4bf2-bfd4-76996df02c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:54.048958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076caea7-49eb-4bf2-bfd4-76996df02c1f None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:54.049240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076caea7-49eb-4bf2-bfd4-76996df02c1f None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:54.049552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076caea7-49eb-4bf2-bfd4-76996df02c1f None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:54.050359 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076caea7-49eb-4bf2-bfd4-76996df02c1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:54.050671 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076caea7-49eb-4bf2-bfd4-76996df02c1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:54.052117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076caea7-49eb-4bf2-bfd4-76996df02c1f None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:54.066056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 665/1272] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:54.075383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daff503-950d-4cf0-b38d-b1f9459d9bdd None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:54.232510 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:54.248350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daff503-950d-4cf0-b38d-b1f9459d9bdd None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:54.305257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443efdb7-4aa3-45a2-ac0d-0498c28fd02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:54.306892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443efdb7-4aa3-45a2-ac0d-0498c28fd02d None admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:54.307377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443efdb7-4aa3-45a2-ac0d-0498c28fd02d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:54.307850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443efdb7-4aa3-45a2-ac0d-0498c28fd02d None admin] PATH_INFO: `/v3/projects/7709322f45334773aaccd8472915ec38/users/f27871b39b9b4edaa512bd5b7467174a/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:54.311628 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443efdb7-4aa3-45a2-ac0d-0498c28fd02d None admin] RBAC: Authorizing `identity:create_grant(project_id=7709322f45334773aaccd8472915ec38, user_id=f27871b39b9b4edaa512bd5b7467174a, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:54.314383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443efdb7-4aa3-45a2-ac0d-0498c28fd02d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:54.337131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 608/1273] 10.222.0.70 () {58 vars in 1439 bytes} [Tue Jul 19 20:38:54 2022] PUT /identity/v3/projects/7709322f45334773aaccd8472915ec38/users/f27871b39b9b4edaa512bd5b7467174a/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:38:56.462918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5063aa4-5746-4e63-88f7-3d05f712fe3f None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:56.463482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5063aa4-5746-4e63-88f7-3d05f712fe3f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:56.463482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5063aa4-5746-4e63-88f7-3d05f712fe3f None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:56.464838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 666/1274] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:38:56.470519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600ce74-3684-45e2-ad09-f5e2c051c8b1 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:56.471050 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600ce74-3684-45e2-ad09-f5e2c051c8b1 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:56.471050 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600ce74-3684-45e2-ad09-f5e2c051c8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:56.488253 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-8600ce74-3684-45e2-ad09-f5e2c051c8b1 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:38:56.489505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8600ce74-3684-45e2-ad09-f5e2c051c8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:38:56.558362 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 609/1275] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:38:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:38:56.582179 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5063aa4-5746-4e63-88f7-3d05f712fe3f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:56.686703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5063aa4-5746-4e63-88f7-3d05f712fe3f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:56.706122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:56.724225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff17a8c-9734-4582-8b7c-c8d098489e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:56.725024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff17a8c-9734-4582-8b7c-c8d098489e0c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:56.725303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff17a8c-9734-4582-8b7c-c8d098489e0c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:56.725610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff17a8c-9734-4582-8b7c-c8d098489e0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:56.736767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff17a8c-9734-4582-8b7c-c8d098489e0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-cff17a8c-9734-4582-8b7c-c8d098489e0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:38:56.744705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:56.745875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:56.747790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 667/1276] 10.222.0.70 () {56 vars in 1096 bytes} [Tue Jul 19 20:38:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:56.752257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600ce74-3684-45e2-ad09-f5e2c051c8b1 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:56.838482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:56.860238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600ce74-3684-45e2-ad09-f5e2c051c8b1 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:56.897028 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fcb32-d11f-4480-9c77-d44f3546fd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:56.897907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fcb32-d11f-4480-9c77-d44f3546fd87 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:56.898917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fcb32-d11f-4480-9c77-d44f3546fd87 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:56.899209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fcb32-d11f-4480-9c77-d44f3546fd87 None admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:56.900265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fcb32-d11f-4480-9c77-d44f3546fd87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:56.900752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fcb32-d11f-4480-9c77-d44f3546fd87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:56.902758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fcb32-d11f-4480-9c77-d44f3546fd87 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:56.915940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 610/1277] 10.222.0.70 () {56 vars in 1099 bytes} [Tue Jul 19 20:38:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:56.923787 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff17a8c-9734-4582-8b7c-c8d098489e0c None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:57.062686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff17a8c-9734-4582-8b7c-c8d098489e0c None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:57.099707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e48c45-3303-4119-8ec4-bb5ee20d0052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:57.100505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e48c45-3303-4119-8ec4-bb5ee20d0052 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:57.100787 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e48c45-3303-4119-8ec4-bb5ee20d0052 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:57.101068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e48c45-3303-4119-8ec4-bb5ee20d0052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:57.112287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e48c45-3303-4119-8ec4-bb5ee20d0052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:57.113757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e48c45-3303-4119-8ec4-bb5ee20d0052 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-f3e48c45-3303-4119-8ec4-bb5ee20d0052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:57.132183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:57.133929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:57.135059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:57.135059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:57.135059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:57.135059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:57.135059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 668/1278] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:57.140109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fcb32-d11f-4480-9c77-d44f3546fd87 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:57.246622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fcb32-d11f-4480-9c77-d44f3546fd87 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:57.286451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee81fe-dd55-4cda-9528-278f7298c52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:57.287126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee81fe-dd55-4cda-9528-278f7298c52d None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:57.287407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee81fe-dd55-4cda-9528-278f7298c52d None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:57.287681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee81fe-dd55-4cda-9528-278f7298c52d None admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:57.288572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee81fe-dd55-4cda-9528-278f7298c52d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:57.289046 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee81fe-dd55-4cda-9528-278f7298c52d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:57.290883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee81fe-dd55-4cda-9528-278f7298c52d None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:57.304251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 611/1279] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:57.311028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e48c45-3303-4119-8ec4-bb5ee20d0052 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:57.358564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:57.452414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e48c45-3303-4119-8ec4-bb5ee20d0052 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:57.490627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cef94-8a49-445a-b2c4-64a46e22fc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:57.491442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cef94-8a49-445a-b2c4-64a46e22fc8b None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:57.491726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cef94-8a49-445a-b2c4-64a46e22fc8b None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:57.492495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cef94-8a49-445a-b2c4-64a46e22fc8b None admin] PATH_INFO: `/v3/users/placement` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:57.501008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867cef94-8a49-445a-b2c4-64a46e22fc8b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:57.502527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867cef94-8a49-445a-b2c4-64a46e22fc8b None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-867cef94-8a49-445a-b2c4-64a46e22fc8b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:38:57.518323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:57.519402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:57.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:57.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:57.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:38:57.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:38:57.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:38:57.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:38:57.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:38:57.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:38:57.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:38:57.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:38:57.520534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 669/1280] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:57.525668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee81fe-dd55-4cda-9528-278f7298c52d None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:57.530127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:57.634269 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee81fe-dd55-4cda-9528-278f7298c52d None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:57.669778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b8a7b-3c0b-4b90-8989-54784da75f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:57.670453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b8a7b-3c0b-4b90-8989-54784da75f60 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:57.670734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b8a7b-3c0b-4b90-8989-54784da75f60 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:57.671008 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b8a7b-3c0b-4b90-8989-54784da75f60 None admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:57.671852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b8a7b-3c0b-4b90-8989-54784da75f60 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:57.672159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b8a7b-3c0b-4b90-8989-54784da75f60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:57.673628 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b8a7b-3c0b-4b90-8989-54784da75f60 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:57.692852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 612/1281] 10.222.0.70 () {56 vars in 1143 bytes} [Tue Jul 19 20:38:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:57.699043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cef94-8a49-445a-b2c4-64a46e22fc8b None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:57.792744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:57.803313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cef94-8a49-445a-b2c4-64a46e22fc8b None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:57.840111 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22130c14-9890-40dc-b025-9d4d56ca24bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:57.840761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22130c14-9890-40dc-b025-9d4d56ca24bc None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:57.841040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22130c14-9890-40dc-b025-9d4d56ca24bc None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:57.841314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22130c14-9890-40dc-b025-9d4d56ca24bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:57.915903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22130c14-9890-40dc-b025-9d4d56ca24bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:57.917537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22130c14-9890-40dc-b025-9d4d56ca24bc None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:58.021106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:58.033323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 670/1282] 10.222.0.70 () {58 vars in 1282 bytes} [Tue Jul 19 20:38:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:38:58.039137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b8a7b-3c0b-4b90-8989-54784da75f60 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:58.111069 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:58.153332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b8a7b-3c0b-4b90-8989-54784da75f60 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:58.192507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc5b29-e4bf-470c-9191-49bd97d9e6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:58.193209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc5b29-e4bf-470c-9191-49bd97d9e6c8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:58.193521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc5b29-e4bf-470c-9191-49bd97d9e6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:58.193802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc5b29-e4bf-470c-9191-49bd97d9e6c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:58.205033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc5b29-e4bf-470c-9191-49bd97d9e6c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:58.206513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc5b29-e4bf-470c-9191-49bd97d9e6c8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-e3bc5b29-e4bf-470c-9191-49bd97d9e6c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:38:58.224058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:58.225187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:58.226218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:38:58.226218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:38:58.226218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:38:58.226218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:58.226218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 613/1283] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:58.231046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22130c14-9890-40dc-b025-9d4d56ca24bc None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:58.332189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22130c14-9890-40dc-b025-9d4d56ca24bc None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:58.365209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fdfd3-235b-42b1-8318-5de890bc5a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:58.365884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fdfd3-235b-42b1-8318-5de890bc5a02 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:58.366168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fdfd3-235b-42b1-8318-5de890bc5a02 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:58.366443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fdfd3-235b-42b1-8318-5de890bc5a02 None admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:58.367715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fdfd3-235b-42b1-8318-5de890bc5a02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:58.368031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fdfd3-235b-42b1-8318-5de890bc5a02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:58.369414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fdfd3-235b-42b1-8318-5de890bc5a02 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:58.382819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 671/1284] 10.222.0.70 () {56 vars in 1109 bytes} [Tue Jul 19 20:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:58.388589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc5b29-e4bf-470c-9191-49bd97d9e6c8 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:58.486004 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc5b29-e4bf-470c-9191-49bd97d9e6c8 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:58.500525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:58.520687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c76202-92af-4672-9039-b93537f0b2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:58.521514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c76202-92af-4672-9039-b93537f0b2c3 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:58.521802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c76202-92af-4672-9039-b93537f0b2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:58.522080 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c76202-92af-4672-9039-b93537f0b2c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:58.528666 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c76202-92af-4672-9039-b93537f0b2c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:58.530140 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c76202-92af-4672-9039-b93537f0b2c3 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application [None req-f9c76202-92af-4672-9039-b93537f0b2c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:38:58.544196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application with Lock( Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:38:58.545215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:38:58.546530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:38:58.546530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:38:58.546530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:38:58.546530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:38:58.546530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:38:58.546530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:38:58.546530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:38:58.546530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: ERROR keystone.server.flask.application Jul 19 20:38:58.546530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 614/1285] 10.222.0.70 () {56 vars in 1111 bytes} [Tue Jul 19 20:38:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:38:58.551282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fdfd3-235b-42b1-8318-5de890bc5a02 None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:58.601152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:38:58.651735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fdfd3-235b-42b1-8318-5de890bc5a02 None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:58.684559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c962a-71bf-4c2d-bb68-c616a728e9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:58.685217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c962a-71bf-4c2d-bb68-c616a728e9ca None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:58.685532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c962a-71bf-4c2d-bb68-c616a728e9ca None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:58.685813 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c962a-71bf-4c2d-bb68-c616a728e9ca None admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:58.686600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c962a-71bf-4c2d-bb68-c616a728e9ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:58.686912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c962a-71bf-4c2d-bb68-c616a728e9ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:58.688333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2c962a-71bf-4c2d-bb68-c616a728e9ca None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:58.700200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 672/1286] 10.222.0.70 () {56 vars in 1148 bytes} [Tue Jul 19 20:38:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:38:58.706633 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c76202-92af-4672-9039-b93537f0b2c3 None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:38:58.804573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c76202-92af-4672-9039-b93537f0b2c3 None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:38:58.837690 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f6f18-2104-4bc6-8062-1e31b2831821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:38:58.838342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f6f18-2104-4bc6-8062-1e31b2831821 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:38:58.838627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f6f18-2104-4bc6-8062-1e31b2831821 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:38:58.838907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f6f18-2104-4bc6-8062-1e31b2831821 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:38:58.839736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f6f18-2104-4bc6-8062-1e31b2831821 None admin] RBAC: Adding query filter params (role.id=85420c68300642aabfd730fb8fef8347, scope.project.id=7709322f45334773aaccd8472915ec38, user.id=f27871b39b9b4edaa512bd5b7467174a) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:38:58.840053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f6f18-2104-4bc6-8062-1e31b2831821 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:38:58.841507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f6f18-2104-4bc6-8062-1e31b2831821 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:38:58.847263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 615/1287] 10.222.0.70 () {56 vars in 1374 bytes} [Tue Jul 19 20:38:58 2022] GET /identity/v3/role_assignments?user.id=f27871b39b9b4edaa512bd5b7467174a&scope.project.id=7709322f45334773aaccd8472915ec38&role.id=85420c68300642aabfd730fb8fef8347 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:01.011830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553472b-f7e8-4b94-a797-1f7070bb4017 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:01.012373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553472b-f7e8-4b94-a797-1f7070bb4017 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:01.012484 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553472b-f7e8-4b94-a797-1f7070bb4017 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:01.014363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 673/1288] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:39:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:01.019778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a1b76-2f43-4dd1-8483-6539350d505c None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:01.020192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a1b76-2f43-4dd1-8483-6539350d505c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:01.020331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a1b76-2f43-4dd1-8483-6539350d505c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:01.037193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0a0a1b76-2f43-4dd1-8483-6539350d505c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:01.038400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0a0a1b76-2f43-4dd1-8483-6539350d505c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:01.067444 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:01.105087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 616/1289] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:39:01.127435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553472b-f7e8-4b94-a797-1f7070bb4017 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:01.227126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553472b-f7e8-4b94-a797-1f7070bb4017 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:01.237926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:01.261415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecdd44-1de0-499b-be86-ee75528f909e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:01.262261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecdd44-1de0-499b-be86-ee75528f909e None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:01.262490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecdd44-1de0-499b-be86-ee75528f909e None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:01.262764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecdd44-1de0-499b-be86-ee75528f909e None admin] PATH_INFO: `/v3/services/placement` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:01.263401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecdd44-1de0-499b-be86-ee75528f909e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:01.264751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecdd44-1de0-499b-be86-ee75528f909e None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-1cecdd44-1de0-499b-be86-ee75528f909e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:01.276993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:01.278601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:39:01.279921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 674/1290] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jul 19 20:39:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:01.284400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a1b76-2f43-4dd1-8483-6539350d505c None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:01.383035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a1b76-2f43-4dd1-8483-6539350d505c None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:01.416456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf8876-e899-4313-9d8a-b659933041a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:01.417109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf8876-e899-4313-9d8a-b659933041a8 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:01.417415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf8876-e899-4313-9d8a-b659933041a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:01.417692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf8876-e899-4313-9d8a-b659933041a8 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:01.418395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf8876-e899-4313-9d8a-b659933041a8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:01.418699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf8876-e899-4313-9d8a-b659933041a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:01.420053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf8876-e899-4313-9d8a-b659933041a8 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:01.424906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 617/1291] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Jul 19 20:39:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:01.431376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecdd44-1de0-499b-be86-ee75528f909e None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:01.529106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecdd44-1de0-499b-be86-ee75528f909e None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:01.562559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede42e6-56e6-49aa-9a02-64b96c8eeaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:01.563214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede42e6-56e6-49aa-9a02-64b96c8eeaec None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:01.563497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede42e6-56e6-49aa-9a02-64b96c8eeaec None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:01.563772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede42e6-56e6-49aa-9a02-64b96c8eeaec None admin] PATH_INFO: `/v3/services` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:01.564470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede42e6-56e6-49aa-9a02-64b96c8eeaec None admin] RBAC: Adding query filter params (type=placement) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:01.564777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede42e6-56e6-49aa-9a02-64b96c8eeaec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:01.566150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede42e6-56e6-49aa-9a02-64b96c8eeaec None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:01.570637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 675/1292] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Jul 19 20:39:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:03.648921 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4b135-90d0-4318-82ad-3a453c5e655c None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:03.649453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4b135-90d0-4318-82ad-3a453c5e655c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:03.649453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4b135-90d0-4318-82ad-3a453c5e655c None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:03.650771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 618/1293] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:03.655923 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98185bf2-c844-4926-8007-93c9c4646018 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:03.656309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98185bf2-c844-4926-8007-93c9c4646018 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:03.656473 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98185bf2-c844-4926-8007-93c9c4646018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:03.673495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-98185bf2-c844-4926-8007-93c9c4646018 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:03.674668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-98185bf2-c844-4926-8007-93c9c4646018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:03.704444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:03.740837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 676/1294] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:39:03.767366 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4b135-90d0-4318-82ad-3a453c5e655c None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:03.807500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:03.884836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4b135-90d0-4318-82ad-3a453c5e655c None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:03.920211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90862384-b77b-48f4-96b1-ee2638d50cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:03.920796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90862384-b77b-48f4-96b1-ee2638d50cc3 None admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:03.921077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90862384-b77b-48f4-96b1-ee2638d50cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:03.921382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90862384-b77b-48f4-96b1-ee2638d50cc3 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:03.922109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90862384-b77b-48f4-96b1-ee2638d50cc3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:03.923490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90862384-b77b-48f4-96b1-ee2638d50cc3 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:03.938482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 619/1295] 10.222.0.70 () {60 vars in 1141 bytes} [Tue Jul 19 20:39:03 2022] POST /identity/v3/services => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:39:05.999172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb47d5-4901-41ab-aed4-36c726bb0b1d None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:05.999745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb47d5-4901-41ab-aed4-36c726bb0b1d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:05.999745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb47d5-4901-41ab-aed4-36c726bb0b1d None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:06.001106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 677/1296] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:06.006302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272cdf4-3313-451f-8ec5-d1904a225241 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:06.006721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272cdf4-3313-451f-8ec5-d1904a225241 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:06.006863 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272cdf4-3313-451f-8ec5-d1904a225241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:06.023251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-b272cdf4-3313-451f-8ec5-d1904a225241 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:06.024428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b272cdf4-3313-451f-8ec5-d1904a225241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:06.094969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 620/1297] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:39:06.119106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb47d5-4901-41ab-aed4-36c726bb0b1d None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:06.221735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb47d5-4901-41ab-aed4-36c726bb0b1d None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:06.255201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39f838-946d-4bce-ac32-84312e76bd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:06.255911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39f838-946d-4bce-ac32-84312e76bd0c None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:06.256191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39f838-946d-4bce-ac32-84312e76bd0c None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:06.256464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39f838-946d-4bce-ac32-84312e76bd0c None admin] PATH_INFO: `/v3/services/placement` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:06.257105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39f838-946d-4bce-ac32-84312e76bd0c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:06.258514 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39f838-946d-4bce-ac32-84312e76bd0c None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-ad39f838-946d-4bce-ac32-84312e76bd0c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:06.270203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:06.271389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:39:06.272418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 678/1298] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jul 19 20:39:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:06.277936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272cdf4-3313-451f-8ec5-d1904a225241 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:06.296356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:06.375838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272cdf4-3313-451f-8ec5-d1904a225241 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:06.409966 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49427b0-188b-4137-8088-86a1a2ebca2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:06.410640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49427b0-188b-4137-8088-86a1a2ebca2e None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:06.410927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49427b0-188b-4137-8088-86a1a2ebca2e None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:06.411208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49427b0-188b-4137-8088-86a1a2ebca2e None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:06.411919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49427b0-188b-4137-8088-86a1a2ebca2e None admin] RBAC: Adding query filter params (name=placement) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:06.412232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49427b0-188b-4137-8088-86a1a2ebca2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:06.413635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49427b0-188b-4137-8088-86a1a2ebca2e None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:06.419383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 621/1299] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Jul 19 20:39:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:06.425379 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39f838-946d-4bce-ac32-84312e76bd0c None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:06.461506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:06.535113 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39f838-946d-4bce-ac32-84312e76bd0c None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:06.567449 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52e114-50fd-42eb-99a9-45fd8c9cfb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:06.568106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52e114-50fd-42eb-99a9-45fd8c9cfb88 None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:06.568391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52e114-50fd-42eb-99a9-45fd8c9cfb88 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:06.568670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52e114-50fd-42eb-99a9-45fd8c9cfb88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:06.569491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52e114-50fd-42eb-99a9-45fd8c9cfb88 None admin] RBAC: Adding query filter params (interface=public, service_id=1c0539ef3f684d5496d5210c3a711b33, region_id=RegionOne) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:06.569810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52e114-50fd-42eb-99a9-45fd8c9cfb88 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:06.571679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52e114-50fd-42eb-99a9-45fd8c9cfb88 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:06.576732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 679/1300] 10.222.0.70 () {56 vars in 1251 bytes} [Tue Jul 19 20:39:06 2022] GET /identity/v3/endpoints?service_id=1c0539ef3f684d5496d5210c3a711b33&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:06.583070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49427b0-188b-4137-8088-86a1a2ebca2e None admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:06.664924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:06.683610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49427b0-188b-4137-8088-86a1a2ebca2e None admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:06.717797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fecba-3169-4375-b4c5-df4f98cc1f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:06.718449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fecba-3169-4375-b4c5-df4f98cc1f05 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:06.718739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fecba-3169-4375-b4c5-df4f98cc1f05 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:06.719017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fecba-3169-4375-b4c5-df4f98cc1f05 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:06.719698 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fecba-3169-4375-b4c5-df4f98cc1f05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:06.721066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85fecba-3169-4375-b4c5-df4f98cc1f05 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:06.727905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 622/1301] 10.222.0.70 () {56 vars in 1087 bytes} [Tue Jul 19 20:39:06 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:39:08.800447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2d5a5-e45e-4954-aa12-b6ad04f70abe None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:08.800971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2d5a5-e45e-4954-aa12-b6ad04f70abe None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:08.801083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2d5a5-e45e-4954-aa12-b6ad04f70abe None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:08.802384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 680/1302] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:08.808148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bc496-4ee9-42f5-9e3d-eadc24f7080e None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:08.808612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bc496-4ee9-42f5-9e3d-eadc24f7080e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:08.808714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bc496-4ee9-42f5-9e3d-eadc24f7080e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:08.825559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-928bc496-4ee9-42f5-9e3d-eadc24f7080e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:08.826771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-928bc496-4ee9-42f5-9e3d-eadc24f7080e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:08.900678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 623/1303] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:39:08.924301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2d5a5-e45e-4954-aa12-b6ad04f70abe None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:09.021487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:09.029263 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2d5a5-e45e-4954-aa12-b6ad04f70abe None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:09.063333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708aebb-4e62-4d17-8f97-31969f54902d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:09.064054 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708aebb-4e62-4d17-8f97-31969f54902d None admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:09.064339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708aebb-4e62-4d17-8f97-31969f54902d None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:09.064618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708aebb-4e62-4d17-8f97-31969f54902d None admin] PATH_INFO: `/v3/services/placement` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:09.065275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708aebb-4e62-4d17-8f97-31969f54902d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:09.066690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708aebb-4e62-4d17-8f97-31969f54902d None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application [None req-d708aebb-4e62-4d17-8f97-31969f54902d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:09.079063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:09.080239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: ERROR keystone.server.flask.application Jul 19 20:39:09.081075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 681/1304] 10.222.0.70 () {56 vars in 1117 bytes} [Tue Jul 19 20:39:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:09.086321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bc496-4ee9-42f5-9e3d-eadc24f7080e None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:09.160038 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:09.183129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bc496-4ee9-42f5-9e3d-eadc24f7080e None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:09.217651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6894d-9d77-4d02-af27-662ad5752963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:09.218336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6894d-9d77-4d02-af27-662ad5752963 None admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:09.218619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6894d-9d77-4d02-af27-662ad5752963 None admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:09.218899 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6894d-9d77-4d02-af27-662ad5752963 None admin] PATH_INFO: `/v3/services` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:09.219635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6894d-9d77-4d02-af27-662ad5752963 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:09.219940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6894d-9d77-4d02-af27-662ad5752963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:09.221407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6894d-9d77-4d02-af27-662ad5752963 None admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:09.228357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 624/1305] 10.222.0.70 () {56 vars in 1116 bytes} [Tue Jul 19 20:39:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:09.234255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708aebb-4e62-4d17-8f97-31969f54902d None admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:09.338709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708aebb-4e62-4d17-8f97-31969f54902d None admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:09.380059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654d257-626f-4ad6-b6a6-a5b3a3d7f9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:09.380723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654d257-626f-4ad6-b6a6-a5b3a3d7f9e6 None admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:09.381008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654d257-626f-4ad6-b6a6-a5b3a3d7f9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:09.381284 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654d257-626f-4ad6-b6a6-a5b3a3d7f9e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:09.382083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654d257-626f-4ad6-b6a6-a5b3a3d7f9e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:09.383446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654d257-626f-4ad6-b6a6-a5b3a3d7f9e6 None admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:09.405780 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 682/1306] 10.222.0.70 () {60 vars in 1144 bytes} [Tue Jul 19 20:39:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:39:19.223858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=30 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:19.400894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:15.942827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e365278-3c8f-4ab7-b654-81e0b7ea4802 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:15.943376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e365278-3c8f-4ab7-b654-81e0b7ea4802 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:15.943376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e365278-3c8f-4ab7-b654-81e0b7ea4802 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:15.945814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 625/1307] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:15.952234 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e87d42-5998-491a-9a10-b98e2928a0f7 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:15.952842 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e87d42-5998-491a-9a10-b98e2928a0f7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:15.953081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e87d42-5998-491a-9a10-b98e2928a0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:16.062857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-84e87d42-5998-491a-9a10-b98e2928a0f7 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:16.064888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-84e87d42-5998-491a-9a10-b98e2928a0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:16.206819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 683/1308] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:16.258345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960399c-906c-4e82-b6f8-5677481444ef None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:16.259032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960399c-906c-4e82-b6f8-5677481444ef None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:16.259454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960399c-906c-4e82-b6f8-5677481444ef None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:16.261406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 626/1309] 10.222.0.70 () {54 vars in 965 bytes} [Tue Jul 19 20:40:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:16.268201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc38ea7-114f-4c4a-8d7b-52080a5ae383 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:16.268761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc38ea7-114f-4c4a-8d7b-52080a5ae383 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:16.269341 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc38ea7-114f-4c4a-8d7b-52080a5ae383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:16.358931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-ccc38ea7-114f-4c4a-8d7b-52080a5ae383 None None] MFA Rules not processed for user `fb186ee2e3c94ca7ad203c9c0e32e498`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:16.360343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ccc38ea7-114f-4c4a-8d7b-52080a5ae383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:16.393301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:16.494773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 684/1310] 10.222.0.70 () {58 vars in 1062 bytes} [Tue Jul 19 20:40:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:16.504346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960399c-906c-4e82-b6f8-5677481444ef None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:16.507206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8960399c-906c-4e82-b6f8-5677481444ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:16.672608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960399c-906c-4e82-b6f8-5677481444ef None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:16.766930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e22c9-cb88-4f76-99b2-c8b175ab641f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:16.767545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e22c9-cb88-4f76-99b2-c8b175ab641f service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:16.767824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e22c9-cb88-4f76-99b2-c8b175ab641f service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:16.768096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e22c9-cb88-4f76-99b2-c8b175ab641f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:16.769909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-320e22c9-cb88-4f76-99b2-c8b175ab641f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:16.836460 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:16.904826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e22c9-cb88-4f76-99b2-c8b175ab641f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:16.906389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e22c9-cb88-4f76-99b2-c8b175ab641f service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:16.941246 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 627/1311] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:40:19.344059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48de44-2493-45a2-91e8-47f3e789c356 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.344595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48de44-2493-45a2-91e8-47f3e789c356 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.344595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48de44-2493-45a2-91e8-47f3e789c356 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.346015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 685/1312] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:19.351318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342d066-cf5d-4eda-a620-28f789a1241e None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.351787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342d066-cf5d-4eda-a620-28f789a1241e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.351893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342d066-cf5d-4eda-a620-28f789a1241e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.433804 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-e342d066-cf5d-4eda-a620-28f789a1241e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:19.435049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e342d066-cf5d-4eda-a620-28f789a1241e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:19.536702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 628/1313] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:19.578700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48de44-2493-45a2-91e8-47f3e789c356 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:19.580514 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-dc48de44-2493-45a2-91e8-47f3e789c356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:19.684524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48de44-2493-45a2-91e8-47f3e789c356 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:19.754257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:19.757854 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac4b6f-ed93-438a-9a55-1e60f9cb2274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:19.758705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac4b6f-ed93-438a-9a55-1e60f9cb2274 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.759155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac4b6f-ed93-438a-9a55-1e60f9cb2274 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.759442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac4b6f-ed93-438a-9a55-1e60f9cb2274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.761170 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d7ac4b6f-ed93-438a-9a55-1e60f9cb2274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:19.868147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac4b6f-ed93-438a-9a55-1e60f9cb2274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:19.869519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac4b6f-ed93-438a-9a55-1e60f9cb2274 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:19.888524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 686/1314] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:40:22.276670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6c883-0727-4f98-b237-e58ea58831a6 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:22.277251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6c883-0727-4f98-b237-e58ea58831a6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:22.277251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6c883-0727-4f98-b237-e58ea58831a6 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:22.278806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 629/1315] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:22.284297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93c3f9-8822-4a22-8f78-29ecf46f115b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:22.284778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93c3f9-8822-4a22-8f78-29ecf46f115b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:22.284937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93c3f9-8822-4a22-8f78-29ecf46f115b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:22.351672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-2a93c3f9-8822-4a22-8f78-29ecf46f115b None None] MFA Rules not processed for user `f9bf4be1c5e94c578c2e02b627c30a56`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:22.352908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2a93c3f9-8822-4a22-8f78-29ecf46f115b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:22.409942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:22.455811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 687/1316] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:22.496158 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6c883-0727-4f98-b237-e58ea58831a6 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:22.497725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-37c6c883-0727-4f98-b237-e58ea58831a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:22.508270 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:22.608524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6c883-0727-4f98-b237-e58ea58831a6 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:22.684896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c0926-8825-44f9-8e2f-768dd3a733f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:22.685580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0926-8825-44f9-8e2f-768dd3a733f5 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:22.685870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0926-8825-44f9-8e2f-768dd3a733f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:22.686151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0926-8825-44f9-8e2f-768dd3a733f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:22.687832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c66c0926-8825-44f9-8e2f-768dd3a733f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:22.752834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:22.804256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c0926-8825-44f9-8e2f-768dd3a733f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:22.805740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c0926-8825-44f9-8e2f-768dd3a733f5 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:22.824976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 630/1317] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:22 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:40:25.489949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f1620-dbbe-4211-b729-7ecbadb7ac41 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.491331 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f1620-dbbe-4211-b729-7ecbadb7ac41 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.491331 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f1620-dbbe-4211-b729-7ecbadb7ac41 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.492470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 688/1318] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:25.499560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc66a9-b025-43a9-8359-c40844f822d0 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.500359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc66a9-b025-43a9-8359-c40844f822d0 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.500648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc66a9-b025-43a9-8359-c40844f822d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.571031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-9adc66a9-b025-43a9-8359-c40844f822d0 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:25.572292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-9adc66a9-b025-43a9-8359-c40844f822d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:25.679382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 631/1319] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:25.718754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f1620-dbbe-4211-b729-7ecbadb7ac41 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:25.720319 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9a2f1620-dbbe-4211-b729-7ecbadb7ac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:25.823890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f1620-dbbe-4211-b729-7ecbadb7ac41 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:25.901916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3016e1-d1ad-4545-9b7c-1a49e4181560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:25.902613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3016e1-d1ad-4545-9b7c-1a49e4181560 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.902898 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3016e1-d1ad-4545-9b7c-1a49e4181560 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.903172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3016e1-d1ad-4545-9b7c-1a49e4181560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.904818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1c3016e1-d1ad-4545-9b7c-1a49e4181560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:25.913709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:26.018471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3016e1-d1ad-4545-9b7c-1a49e4181560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:26.019813 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3016e1-d1ad-4545-9b7c-1a49e4181560 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:26.038640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 689/1320] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:40:28.813995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9bf5e-b0d2-4452-b43d-511d798e10c9 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:28.814586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9bf5e-b0d2-4452-b43d-511d798e10c9 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:28.814586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9bf5e-b0d2-4452-b43d-511d798e10c9 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:28.816027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 632/1321] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:28.821851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f574f4-8d2e-47d7-bab6-6cf40cbdb7e8 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:28.822370 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f574f4-8d2e-47d7-bab6-6cf40cbdb7e8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:28.822485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f574f4-8d2e-47d7-bab6-6cf40cbdb7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:28.891566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-c4f574f4-8d2e-47d7-bab6-6cf40cbdb7e8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:28.893205 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c4f574f4-8d2e-47d7-bab6-6cf40cbdb7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:28.961000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:29.007567 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 690/1322] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:29.062186 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba07e1-179d-4c8c-b528-b3711bc39f24 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.062577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba07e1-179d-4c8c-b528-b3711bc39f24 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.062758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba07e1-179d-4c8c-b528-b3711bc39f24 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.064363 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 633/1323] 10.222.0.70 () {54 vars in 965 bytes} [Tue Jul 19 20:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:29.071835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09012b-0be9-418b-ab24-719824e24585 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.072105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09012b-0be9-418b-ab24-719824e24585 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.072391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09012b-0be9-418b-ab24-719824e24585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.162305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0a09012b-0be9-418b-ab24-719824e24585 None None] MFA Rules not processed for user `fb186ee2e3c94ca7ad203c9c0e32e498`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:29.163520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0a09012b-0be9-418b-ab24-719824e24585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:29.250159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 691/1324] 10.222.0.70 () {58 vars in 1062 bytes} [Tue Jul 19 20:40:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:29.259766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba07e1-179d-4c8c-b528-b3711bc39f24 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:29.261320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-66ba07e1-179d-4c8c-b528-b3711bc39f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:29.282874 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:29.385224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba07e1-179d-4c8c-b528-b3711bc39f24 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:29.481181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e6a52-9bd4-46cb-a8e6-38ae3cca43e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:29.482678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6a52-9bd4-46cb-a8e6-38ae3cca43e6 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.483401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6a52-9bd4-46cb-a8e6-38ae3cca43e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.484047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e6a52-9bd4-46cb-a8e6-38ae3cca43e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.487096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-181e6a52-9bd4-46cb-a8e6-38ae3cca43e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:29.583501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:29.635712 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6a52-9bd4-46cb-a8e6-38ae3cca43e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:29.637202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e6a52-9bd4-46cb-a8e6-38ae3cca43e6 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:29.659457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 634/1325] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:40:32.040766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d227e4-691a-4d4a-9715-78c713babc1f None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:32.041382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d227e4-691a-4d4a-9715-78c713babc1f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:32.041382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d227e4-691a-4d4a-9715-78c713babc1f None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:32.042755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 692/1326] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:32.048443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85975004-59e6-4575-b232-183447026ea6 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:32.048884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85975004-59e6-4575-b232-183447026ea6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:32.049032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85975004-59e6-4575-b232-183447026ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:32.117059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-85975004-59e6-4575-b232-183447026ea6 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:32.118301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-85975004-59e6-4575-b232-183447026ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:32.232178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 635/1327] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:32.269605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d227e4-691a-4d4a-9715-78c713babc1f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:32.271175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-11d227e4-691a-4d4a-9715-78c713babc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:32.335153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:32.380086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d227e4-691a-4d4a-9715-78c713babc1f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:32.456636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29363881-fdbe-4048-9dc6-d4a6e34c0e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:32.457190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29363881-fdbe-4048-9dc6-d4a6e34c0e4a service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:32.457505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29363881-fdbe-4048-9dc6-d4a6e34c0e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:32.457782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29363881-fdbe-4048-9dc6-d4a6e34c0e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:32.459462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-29363881-fdbe-4048-9dc6-d4a6e34c0e4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:32.572322 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:32.574673 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29363881-fdbe-4048-9dc6-d4a6e34c0e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:32.576397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29363881-fdbe-4048-9dc6-d4a6e34c0e4a service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:32.595875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 693/1328] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:40:34.712967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1109b8-1e41-47c5-8f9e-f0c7b035ad1f None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.713512 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1109b8-1e41-47c5-8f9e-f0c7b035ad1f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.713629 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1109b8-1e41-47c5-8f9e-f0c7b035ad1f None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.714988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 636/1329] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:34.720193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7feb1-328d-402b-9087-ed089c80e567 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.720616 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7feb1-328d-402b-9087-ed089c80e567 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.720802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7feb1-328d-402b-9087-ed089c80e567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.785643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-25f7feb1-328d-402b-9087-ed089c80e567 None None] MFA Rules not processed for user `f9bf4be1c5e94c578c2e02b627c30a56`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:34.786849 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-25f7feb1-328d-402b-9087-ed089c80e567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:34.918044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 694/1330] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:34.956992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1109b8-1e41-47c5-8f9e-f0c7b035ad1f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:34.958553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0e1109b8-1e41-47c5-8f9e-f0c7b035ad1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:34.988441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:35.062405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1109b8-1e41-47c5-8f9e-f0c7b035ad1f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:35.135449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c865e1f-5715-4def-b180-55e18ecb8a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:35.136034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865e1f-5715-4def-b180-55e18ecb8a8f service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:35.136382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865e1f-5715-4def-b180-55e18ecb8a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:35.136663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865e1f-5715-4def-b180-55e18ecb8a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:35.138345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2c865e1f-5715-4def-b180-55e18ecb8a8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:35.216093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:35.251319 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c865e1f-5715-4def-b180-55e18ecb8a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:35.252760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c865e1f-5715-4def-b180-55e18ecb8a8f service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:35.272138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 637/1331] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:34 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:40:37.646227 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf6bfa-20fb-479f-8ea3-8e0be6ed3a9e None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:37.647284 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf6bfa-20fb-479f-8ea3-8e0be6ed3a9e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:37.647284 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf6bfa-20fb-479f-8ea3-8e0be6ed3a9e None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:37.648635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 695/1332] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:37.654959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565eaf-c2a2-48a7-a5a9-d1a5a92828df None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:37.655578 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565eaf-c2a2-48a7-a5a9-d1a5a92828df None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:37.655578 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565eaf-c2a2-48a7-a5a9-d1a5a92828df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:37.738834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a5565eaf-c2a2-48a7-a5a9-d1a5a92828df None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:37.740100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a5565eaf-c2a2-48a7-a5a9-d1a5a92828df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:37.862348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 638/1333] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:37.898881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf6bfa-20fb-479f-8ea3-8e0be6ed3a9e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:37.900381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9ccf6bfa-20fb-479f-8ea3-8e0be6ed3a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:37.986781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:38.029127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf6bfa-20fb-479f-8ea3-8e0be6ed3a9e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:38.115818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437ae16-6af7-4480-a739-66346fc23684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:38.116488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437ae16-6af7-4480-a739-66346fc23684 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.116773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437ae16-6af7-4480-a739-66346fc23684 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.117049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437ae16-6af7-4480-a739-66346fc23684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.118889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6437ae16-6af7-4480-a739-66346fc23684 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:38.259046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437ae16-6af7-4480-a739-66346fc23684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:38.260453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437ae16-6af7-4480-a739-66346fc23684 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:38.272390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:38.282663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 696/1334] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:40:40.572532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea471dc1-b185-41aa-96c4-d79f2b337460 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:40.573114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea471dc1-b185-41aa-96c4-d79f2b337460 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:40.573519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea471dc1-b185-41aa-96c4-d79f2b337460 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:40.576034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 639/1335] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:40.582019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dee06-f346-4771-8ca5-34d750fdafbb None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:40.582519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dee06-f346-4771-8ca5-34d750fdafbb None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:40.582738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dee06-f346-4771-8ca5-34d750fdafbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:40.659243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-744dee06-f346-4771-8ca5-34d750fdafbb None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:40.660343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-744dee06-f346-4771-8ca5-34d750fdafbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:40.775390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 697/1336] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:40.822773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea471dc1-b185-41aa-96c4-d79f2b337460 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:40.824365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ea471dc1-b185-41aa-96c4-d79f2b337460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:40.877889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:40.967184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea471dc1-b185-41aa-96c4-d79f2b337460 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:41.090327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad418b8-73d6-414e-ab33-ea9d43dd86e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:41.091450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad418b8-73d6-414e-ab33-ea9d43dd86e2 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:41.091931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad418b8-73d6-414e-ab33-ea9d43dd86e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:41.092437 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad418b8-73d6-414e-ab33-ea9d43dd86e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:41.095610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-dad418b8-73d6-414e-ab33-ea9d43dd86e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:41.216383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:41.246474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad418b8-73d6-414e-ab33-ea9d43dd86e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:41.248637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad418b8-73d6-414e-ab33-ea9d43dd86e2 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:41.270159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 640/1337] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:40:44.338619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f16a9-2235-4abb-bc43-3d71c0368eb5 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:44.339261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f16a9-2235-4abb-bc43-3d71c0368eb5 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:44.339261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f16a9-2235-4abb-bc43-3d71c0368eb5 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:44.340921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 698/1338] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:44.346874 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e25a82-02c0-40b5-8589-ffedc9206fcf None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:44.347403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e25a82-02c0-40b5-8589-ffedc9206fcf None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:44.347565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e25a82-02c0-40b5-8589-ffedc9206fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:44.424179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-25e25a82-02c0-40b5-8589-ffedc9206fcf None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:44.425531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-25e25a82-02c0-40b5-8589-ffedc9206fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:44.545922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 641/1339] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:44.587803 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f16a9-2235-4abb-bc43-3d71c0368eb5 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:44.589672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-310f16a9-2235-4abb-bc43-3d71c0368eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:44.692758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:44.732180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f16a9-2235-4abb-bc43-3d71c0368eb5 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:44.823346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21ec36-ba05-451b-adc0-cda10388f3c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:44.824046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21ec36-ba05-451b-adc0-cda10388f3c3 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:44.824325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21ec36-ba05-451b-adc0-cda10388f3c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:44.824622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21ec36-ba05-451b-adc0-cda10388f3c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:44.826771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ff21ec36-ba05-451b-adc0-cda10388f3c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:44.966439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff21ec36-ba05-451b-adc0-cda10388f3c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:44.968010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff21ec36-ba05-451b-adc0-cda10388f3c3 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:44.990043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 699/1340] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:40:49.594015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096c89-a275-4d3d-88be-e10737775a10 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:49.594481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096c89-a275-4d3d-88be-e10737775a10 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:49.594986 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096c89-a275-4d3d-88be-e10737775a10 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:49.597542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 642/1341] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:49.604253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cea18-5d4e-478a-b5b1-4c87a5b0cd3c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:49.604852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cea18-5d4e-478a-b5b1-4c87a5b0cd3c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:49.605257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cea18-5d4e-478a-b5b1-4c87a5b0cd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:49.613186 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:49.718488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-458cea18-5d4e-478a-b5b1-4c87a5b0cd3c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:49.720099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-458cea18-5d4e-478a-b5b1-4c87a5b0cd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:49.839517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 700/1342] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:40:49.891522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e096c89-a275-4d3d-88be-e10737775a10 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:49.893391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7e096c89-a275-4d3d-88be-e10737775a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:49.958910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:50.029065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e096c89-a275-4d3d-88be-e10737775a10 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:50.119429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26618315-879b-4c7b-805e-bd36dc7d30a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:50.120161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26618315-879b-4c7b-805e-bd36dc7d30a3 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:50.120457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26618315-879b-4c7b-805e-bd36dc7d30a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:50.120732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26618315-879b-4c7b-805e-bd36dc7d30a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:50.122587 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-26618315-879b-4c7b-805e-bd36dc7d30a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:50.234286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:50.255431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26618315-879b-4c7b-805e-bd36dc7d30a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:50.257017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26618315-879b-4c7b-805e-bd36dc7d30a3 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:50.277830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 643/1343] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:40:53.143942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad8f2c-d239-4961-9308-88ad07cfd5f0 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.144582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad8f2c-d239-4961-9308-88ad07cfd5f0 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.144582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad8f2c-d239-4961-9308-88ad07cfd5f0 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.146307 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 701/1344] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:53.154704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84820c-fa10-43da-8efe-8927b67ea9a4 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.155326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84820c-fa10-43da-8efe-8927b67ea9a4 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.155326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84820c-fa10-43da-8efe-8927b67ea9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.236005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-5a84820c-fa10-43da-8efe-8927b67ea9a4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:53.237293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5a84820c-fa10-43da-8efe-8927b67ea9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:53.382342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 644/1345] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:40:53.426375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad8f2c-d239-4961-9308-88ad07cfd5f0 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:53.428618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4cad8f2c-d239-4961-9308-88ad07cfd5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:53.550963 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:53.576372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad8f2c-d239-4961-9308-88ad07cfd5f0 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:53.695436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2736b6-7ac8-4b90-8a6b-3c72ed4ab968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:53.696070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2736b6-7ac8-4b90-8a6b-3c72ed4ab968 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.696360 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2736b6-7ac8-4b90-8a6b-3c72ed4ab968 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.696645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2736b6-7ac8-4b90-8a6b-3c72ed4ab968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.699382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-be2736b6-7ac8-4b90-8a6b-3c72ed4ab968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:53.864037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2736b6-7ac8-4b90-8a6b-3c72ed4ab968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:53.866131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2736b6-7ac8-4b90-8a6b-3c72ed4ab968 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:53.885948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 702/1346] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:40:58.875159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2af6-5fee-41f3-9dd9-8a141b7d761f None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.875742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2af6-5fee-41f3-9dd9-8a141b7d761f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.875742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2af6-5fee-41f3-9dd9-8a141b7d761f None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.877706 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 645/1347] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:40:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:58.886242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4359-a842-4ef9-9bab-58fc016b62e7 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.886934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4359-a842-4ef9-9bab-58fc016b62e7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.886934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4359-a842-4ef9-9bab-58fc016b62e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.914775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:58.988796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-8fcf4359-a842-4ef9-9bab-58fc016b62e7 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:58.990736 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8fcf4359-a842-4ef9-9bab-58fc016b62e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:59.168853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 703/1348] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:40:59.214448 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad2af6-5fee-41f3-9dd9-8a141b7d761f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:59.216101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a5ad2af6-5fee-41f3-9dd9-8a141b7d761f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:59.307166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:59.378895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad2af6-5fee-41f3-9dd9-8a141b7d761f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:59.519129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb71fee-9ea8-476c-a26a-91d84358e16a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:59.520160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb71fee-9ea8-476c-a26a-91d84358e16a service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:59.520634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb71fee-9ea8-476c-a26a-91d84358e16a service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:59.521088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb71fee-9ea8-476c-a26a-91d84358e16a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:59.523640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5eb71fee-9ea8-476c-a26a-91d84358e16a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:59.694549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:59.699774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb71fee-9ea8-476c-a26a-91d84358e16a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:59.701837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb71fee-9ea8-476c-a26a-91d84358e16a service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:59.728386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 646/1349] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:40:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:41:05.931735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7dfb3-1d57-4bbf-b10f-e1a489a8fd8b None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.932550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7dfb3-1d57-4bbf-b10f-e1a489a8fd8b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.932550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7dfb3-1d57-4bbf-b10f-e1a489a8fd8b None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.937123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 704/1350] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 20:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:05.944728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b3623-4e38-42a2-8bee-1eb73139c5f3 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.946136 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b3623-4e38-42a2-8bee-1eb73139c5f3 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.946843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b3623-4e38-42a2-8bee-1eb73139c5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:06.122384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ff3b3623-4e38-42a2-8bee-1eb73139c5f3 None None] MFA Rules not processed for user `f27871b39b9b4edaa512bd5b7467174a`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:06.124774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ff3b3623-4e38-42a2-8bee-1eb73139c5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:06.403523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 647/1351] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 20:41:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:09.169746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=59 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:10.947642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a436bbc-32bc-4954-8ab9-b9e6d4894d8d None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.948253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a436bbc-32bc-4954-8ab9-b9e6d4894d8d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.948253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a436bbc-32bc-4954-8ab9-b9e6d4894d8d None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.949908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 705/1352] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 20:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:10.955425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfc663-40a9-4913-b159-ab14809ef468 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.956163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfc663-40a9-4913-b159-ab14809ef468 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.956246 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfc663-40a9-4913-b159-ab14809ef468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:11.017296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:11.050236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-88dfc663-40a9-4913-b159-ab14809ef468 None None] MFA Rules not processed for user `f27871b39b9b4edaa512bd5b7467174a`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:11.051686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-88dfc663-40a9-4913-b159-ab14809ef468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:11.175959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 648/1353] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 20:41:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:12.665851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2510d04-5948-4d92-8254-562a9ec1198f None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:12.666367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2510d04-5948-4d92-8254-562a9ec1198f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:12.666367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2510d04-5948-4d92-8254-562a9ec1198f None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:12.667702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 706/1354] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 20:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:12.673108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45228968-6ac4-4492-92ae-4f1ff7dfdea9 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:12.673633 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45228968-6ac4-4492-92ae-4f1ff7dfdea9 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:12.673730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45228968-6ac4-4492-92ae-4f1ff7dfdea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:12.763111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-45228968-6ac4-4492-92ae-4f1ff7dfdea9 None None] MFA Rules not processed for user `f27871b39b9b4edaa512bd5b7467174a`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:12.764404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-45228968-6ac4-4492-92ae-4f1ff7dfdea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:12.827918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:12.924932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 649/1355] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 20:41:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:16.451323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad665e1-9dbf-4644-853a-1badcffbe6ed None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:16.453050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad665e1-9dbf-4644-853a-1badcffbe6ed None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:16.453700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad665e1-9dbf-4644-853a-1badcffbe6ed None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.455867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 707/1356] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:16.463067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d293d7a-5040-43bc-bbfc-0dba5cda6802 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:16.463652 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d293d7a-5040-43bc-bbfc-0dba5cda6802 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:16.464226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d293d7a-5040-43bc-bbfc-0dba5cda6802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.594657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-9d293d7a-5040-43bc-bbfc-0dba5cda6802 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:16.595807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-9d293d7a-5040-43bc-bbfc-0dba5cda6802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:16.743723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 650/1357] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:16.798495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63185cf-0135-4ed0-816c-6719da371b2a None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:16.799374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63185cf-0135-4ed0-816c-6719da371b2a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:16.800065 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63185cf-0135-4ed0-816c-6719da371b2a None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.802044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 708/1358] 10.222.0.70 () {54 vars in 962 bytes} [Tue Jul 19 20:41:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:16.810798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea10bc-2ac9-4f98-af97-138655258bec None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:16.811089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea10bc-2ac9-4f98-af97-138655258bec None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:16.811352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea10bc-2ac9-4f98-af97-138655258bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.827922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:16.900998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-fbea10bc-2ac9-4f98-af97-138655258bec None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:16.903015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-fbea10bc-2ac9-4f98-af97-138655258bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:17.046289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 651/1359] 10.222.0.70 () {58 vars in 1059 bytes} [Tue Jul 19 20:41:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:17.054625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63185cf-0135-4ed0-816c-6719da371b2a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:17.056266 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d63185cf-0135-4ed0-816c-6719da371b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:17.220188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63185cf-0135-4ed0-816c-6719da371b2a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:17.299453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:17.345655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5c615-98f2-4380-9560-a1f0bb608ff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:17.346315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5c615-98f2-4380-9560-a1f0bb608ff1 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:17.346602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5c615-98f2-4380-9560-a1f0bb608ff1 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:17.346883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5c615-98f2-4380-9560-a1f0bb608ff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:17.348579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5dd5c615-98f2-4380-9560-a1f0bb608ff1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:17.457235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5c615-98f2-4380-9560-a1f0bb608ff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:17.459686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5c615-98f2-4380-9560-a1f0bb608ff1 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:17.481467 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 709/1360] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:41:19.962076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc41cda-ceeb-45d8-ab7a-8e4b803a4f38 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:19.962613 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc41cda-ceeb-45d8-ab7a-8e4b803a4f38 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:19.962613 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc41cda-ceeb-45d8-ab7a-8e4b803a4f38 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:19.964144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 652/1361] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:19.969339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9b7ee-c6fd-41e5-8d04-916c79a60fc2 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:19.969946 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9b7ee-c6fd-41e5-8d04-916c79a60fc2 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:19.970157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9b7ee-c6fd-41e5-8d04-916c79a60fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.049155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a1d9b7ee-c6fd-41e5-8d04-916c79a60fc2 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:20.050368 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a1d9b7ee-c6fd-41e5-8d04-916c79a60fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:20.095418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:20.204075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 710/1362] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:20.254333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc41cda-ceeb-45d8-ab7a-8e4b803a4f38 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:20.257098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-dcc41cda-ceeb-45d8-ab7a-8e4b803a4f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:20.335623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:20.483902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc41cda-ceeb-45d8-ab7a-8e4b803a4f38 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:20.631003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d404aa-5040-4197-863f-6c819de31f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:20.631992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d404aa-5040-4197-863f-6c819de31f83 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:20.631992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d404aa-5040-4197-863f-6c819de31f83 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:20.632601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d404aa-5040-4197-863f-6c819de31f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.634210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-63d404aa-5040-4197-863f-6c819de31f83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:20.661986 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:20.754711 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d404aa-5040-4197-863f-6c819de31f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:20.757558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d404aa-5040-4197-863f-6c819de31f83 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:20.794225 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 653/1363] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:41:23.324357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd978c-e1f3-4999-a726-b717f6a228aa None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.324944 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd978c-e1f3-4999-a726-b717f6a228aa None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.324944 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd978c-e1f3-4999-a726-b717f6a228aa None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.326320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 711/1364] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:23.332402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd150d7-38e3-4e77-ac13-9d6ea2f27a40 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.332910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd150d7-38e3-4e77-ac13-9d6ea2f27a40 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.333073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd150d7-38e3-4e77-ac13-9d6ea2f27a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.415880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-3fd150d7-38e3-4e77-ac13-9d6ea2f27a40 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:23.417114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3fd150d7-38e3-4e77-ac13-9d6ea2f27a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:23.525313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:23.533022 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 654/1365] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:23.588075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63a8b1-941d-43a6-a52a-bdafe899e9ac None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.588701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63a8b1-941d-43a6-a52a-bdafe899e9ac None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.588981 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63a8b1-941d-43a6-a52a-bdafe899e9ac None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.590924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 712/1366] 10.222.0.70 () {54 vars in 962 bytes} [Tue Jul 19 20:41:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:23.596885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec87580-ed46-4e17-90fc-ec0a2c6309fc None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.597248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec87580-ed46-4e17-90fc-ec0a2c6309fc None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.597641 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec87580-ed46-4e17-90fc-ec0a2c6309fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.691567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-cec87580-ed46-4e17-90fc-ec0a2c6309fc None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:23.693390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-cec87580-ed46-4e17-90fc-ec0a2c6309fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:23.839534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 655/1367] 10.222.0.70 () {58 vars in 1059 bytes} [Tue Jul 19 20:41:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:41:23.848602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63a8b1-941d-43a6-a52a-bdafe899e9ac None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:23.850736 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-da63a8b1-941d-43a6-a52a-bdafe899e9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:24.031872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63a8b1-941d-43a6-a52a-bdafe899e9ac None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:24.054508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:24.156842 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc82a1e-c352-41fe-862d-fd171fb43d88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:24.157435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc82a1e-c352-41fe-862d-fd171fb43d88 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:24.157723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc82a1e-c352-41fe-862d-fd171fb43d88 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:24.158006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc82a1e-c352-41fe-862d-fd171fb43d88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:24.159658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-dfc82a1e-c352-41fe-862d-fd171fb43d88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:24.258577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc82a1e-c352-41fe-862d-fd171fb43d88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:24.260185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc82a1e-c352-41fe-862d-fd171fb43d88 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:24.280934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 713/1368] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:41:25.362112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b8e7c-7722-4ec6-a83f-431a1bda1f63 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:25.363583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b8e7c-7722-4ec6-a83f-431a1bda1f63 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:25.363583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b8e7c-7722-4ec6-a83f-431a1bda1f63 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:25.364365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 656/1369] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:25.371124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b9499-1ac9-44f1-b957-7d1ab37aabf1 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:25.371124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b9499-1ac9-44f1-b957-7d1ab37aabf1 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:25.371124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b9499-1ac9-44f1-b957-7d1ab37aabf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:25.408642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:25.453345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-7e6b9499-1ac9-44f1-b957-7d1ab37aabf1 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:25.454808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-7e6b9499-1ac9-44f1-b957-7d1ab37aabf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:25.593775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 714/1370] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:25.939450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485c75c-b5bf-45a8-b212-956f780366c0 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:25.939450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485c75c-b5bf-45a8-b212-956f780366c0 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:25.939450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485c75c-b5bf-45a8-b212-956f780366c0 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:25.946595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 657/1371] 10.222.0.70 () {54 vars in 964 bytes} [Tue Jul 19 20:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:25.947501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5903f-60db-42fb-b1cb-9c75b8bd7ddf None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:25.947501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5903f-60db-42fb-b1cb-9c75b8bd7ddf None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:25.947501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5903f-60db-42fb-b1cb-9c75b8bd7ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.046740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-adb5903f-60db-42fb-b1cb-9c75b8bd7ddf None None] MFA Rules not processed for user `a75951e9512c420992a4fd6c1a6b1176`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:26.047939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-adb5903f-60db-42fb-b1cb-9c75b8bd7ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:26.108889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:26.143419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 715/1372] 10.222.0.70 () {58 vars in 1061 bytes} [Tue Jul 19 20:41:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:26.152499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485c75c-b5bf-45a8-b212-956f780366c0 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:26.155536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4485c75c-b5bf-45a8-b212-956f780366c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:26.227364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:26.304994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485c75c-b5bf-45a8-b212-956f780366c0 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:26.428753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a2bdd-3bef-40f1-9481-2735c9ba5619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:26.429808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a2bdd-3bef-40f1-9481-2735c9ba5619 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.429808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a2bdd-3bef-40f1-9481-2735c9ba5619 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.430725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a2bdd-3bef-40f1-9481-2735c9ba5619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.432515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bd0a2bdd-3bef-40f1-9481-2735c9ba5619 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:26.467184 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305ac19-b167-4e2b-a9d8-1ff6554c0e6d None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.467728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305ac19-b167-4e2b-a9d8-1ff6554c0e6d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.468107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305ac19-b167-4e2b-a9d8-1ff6554c0e6d None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.471237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 716/1373] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:26.477272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2b20f-4ef5-4d90-9391-67ff5453cd76 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.477653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2b20f-4ef5-4d90-9391-67ff5453cd76 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.477942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2b20f-4ef5-4d90-9391-67ff5453cd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.574984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:26.579165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-80e2b20f-4ef5-4d90-9391-67ff5453cd76 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:26.581997 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-80e2b20f-4ef5-4d90-9391-67ff5453cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:26.593301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a2bdd-3bef-40f1-9481-2735c9ba5619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:26.595693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a2bdd-3bef-40f1-9481-2735c9ba5619 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:26.615205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 658/1374] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:41:26.743446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 717/1375] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:41:26.791027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a2bdd-3bef-40f1-9481-2735c9ba5619 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:26.793273 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bd0a2bdd-3bef-40f1-9481-2735c9ba5619 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:26.910105 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a2bdd-3bef-40f1-9481-2735c9ba5619 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:27.039315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:27.051786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa137bae-9c8c-4660-b036-c5e1f4bbfb65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:27.052436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa137bae-9c8c-4660-b036-c5e1f4bbfb65 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:27.052720 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa137bae-9c8c-4660-b036-c5e1f4bbfb65 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:27.052993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa137bae-9c8c-4660-b036-c5e1f4bbfb65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:27.054734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-aa137bae-9c8c-4660-b036-c5e1f4bbfb65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:27.160802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa137bae-9c8c-4660-b036-c5e1f4bbfb65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:27.162471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa137bae-9c8c-4660-b036-c5e1f4bbfb65 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:27.182326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 659/1376] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:41:29.373170 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dea715-968b-4284-bea4-ffe39db7dd0a None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:29.373922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dea715-968b-4284-bea4-ffe39db7dd0a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:29.373922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dea715-968b-4284-bea4-ffe39db7dd0a None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:29.375529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 718/1377] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:29.384377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723988f-e7e7-45aa-9e6f-4a44b513e787 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:29.385121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723988f-e7e7-45aa-9e6f-4a44b513e787 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:29.385279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723988f-e7e7-45aa-9e6f-4a44b513e787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:29.506376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-b723988f-e7e7-45aa-9e6f-4a44b513e787 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:29.507630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b723988f-e7e7-45aa-9e6f-4a44b513e787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:29.586116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:29.640583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 660/1378] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:29.688728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dea715-968b-4284-bea4-ffe39db7dd0a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:29.690362 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b6dea715-968b-4284-bea4-ffe39db7dd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:29.802132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:30.020093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dea715-968b-4284-bea4-ffe39db7dd0a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:30.193082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c4f1f-d7a0-4333-9656-bf58d9bc7b68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:30.193804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c4f1f-d7a0-4333-9656-bf58d9bc7b68 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.194088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c4f1f-d7a0-4333-9656-bf58d9bc7b68 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.194365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c4f1f-d7a0-4333-9656-bf58d9bc7b68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.196146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-cb0c4f1f-d7a0-4333-9656-bf58d9bc7b68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:30.211647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:30.322779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c4f1f-d7a0-4333-9656-bf58d9bc7b68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:30.329497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c4f1f-d7a0-4333-9656-bf58d9bc7b68 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:30.379891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 719/1379] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:41:32.875652 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0e70f-a757-4584-b424-e1192400d6ef None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:32.876276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0e70f-a757-4584-b424-e1192400d6ef None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:32.876669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0e70f-a757-4584-b424-e1192400d6ef None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:32.879192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 661/1380] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:32.887069 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c0d7f-9a5d-49ae-b3fe-be8a25bbb519 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:32.887926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c0d7f-9a5d-49ae-b3fe-be8a25bbb519 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:32.887926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c0d7f-9a5d-49ae-b3fe-be8a25bbb519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:33.005645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-7a2c0d7f-9a5d-49ae-b3fe-be8a25bbb519 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:33.007705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-7a2c0d7f-9a5d-49ae-b3fe-be8a25bbb519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:33.172278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:33.197697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 720/1381] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:33.250250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0e70f-a757-4584-b424-e1192400d6ef None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:33.253659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-f4b0e70f-a757-4584-b424-e1192400d6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:33.520010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0e70f-a757-4584-b424-e1192400d6ef None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:33.594764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:33.706385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418c9f2-0225-465c-a36c-1bfea7279c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:33.707350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418c9f2-0225-465c-a36c-1bfea7279c0b service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:33.707795 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418c9f2-0225-465c-a36c-1bfea7279c0b service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:33.708246 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418c9f2-0225-465c-a36c-1bfea7279c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:33.710676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b418c9f2-0225-465c-a36c-1bfea7279c0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:33.862499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418c9f2-0225-465c-a36c-1bfea7279c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:33.866454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418c9f2-0225-465c-a36c-1bfea7279c0b service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:33.902200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 662/1382] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:41:36.514335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f5bbe-99ef-4282-b433-ae4d235d867d None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.514930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f5bbe-99ef-4282-b433-ae4d235d867d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.515038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f5bbe-99ef-4282-b433-ae4d235d867d None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.516649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 721/1383] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:36.524054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933647a0-77c6-48cb-be86-0b4660bb58b2 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.524667 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933647a0-77c6-48cb-be86-0b4660bb58b2 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.526624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933647a0-77c6-48cb-be86-0b4660bb58b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.662386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-933647a0-77c6-48cb-be86-0b4660bb58b2 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:36.664907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-933647a0-77c6-48cb-be86-0b4660bb58b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:36.668878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:36.814719 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 663/1384] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:36.875263 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f5bbe-99ef-4282-b433-ae4d235d867d None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:36.876992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-150f5bbe-99ef-4282-b433-ae4d235d867d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:37.144055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f5bbe-99ef-4282-b433-ae4d235d867d None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:37.241777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:37.337792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441d9f9-68c8-4154-b458-585411a4f886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:37.339053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441d9f9-68c8-4154-b458-585411a4f886 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:37.339592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441d9f9-68c8-4154-b458-585411a4f886 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:37.340123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441d9f9-68c8-4154-b458-585411a4f886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:37.343035 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8441d9f9-68c8-4154-b458-585411a4f886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:37.513710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441d9f9-68c8-4154-b458-585411a4f886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:37.515813 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441d9f9-68c8-4154-b458-585411a4f886 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:37.548770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 722/1385] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:41:40.261473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f576e0-3330-4f1f-960e-bf8350e32f03 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.262698 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f576e0-3330-4f1f-960e-bf8350e32f03 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.262698 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f576e0-3330-4f1f-960e-bf8350e32f03 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.264484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 664/1386] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:40.274269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a1152-2718-48ea-b71b-02006a96912b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.275748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a1152-2718-48ea-b71b-02006a96912b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.275748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a1152-2718-48ea-b71b-02006a96912b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.374583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-4d0a1152-2718-48ea-b71b-02006a96912b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:40.376831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4d0a1152-2718-48ea-b71b-02006a96912b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:40.418479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:40.529950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 723/1387] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:40.595887 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f576e0-3330-4f1f-960e-bf8350e32f03 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:40.598290 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-14f576e0-3330-4f1f-960e-bf8350e32f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:40.809428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:40.838858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f576e0-3330-4f1f-960e-bf8350e32f03 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:41.043832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28c28f-23e9-4463-90d5-cba5b904eaf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:41.044961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28c28f-23e9-4463-90d5-cba5b904eaf6 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:41.045477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28c28f-23e9-4463-90d5-cba5b904eaf6 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:41.045835 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28c28f-23e9-4463-90d5-cba5b904eaf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:41.047771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3e28c28f-23e9-4463-90d5-cba5b904eaf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:41.173736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e28c28f-23e9-4463-90d5-cba5b904eaf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:41.175623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e28c28f-23e9-4463-90d5-cba5b904eaf6 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:41.182748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:41.201134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 665/1388] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:41:43.271315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367548d-41c0-42df-b38f-1b28d893589b None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.272748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367548d-41c0-42df-b38f-1b28d893589b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.272748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367548d-41c0-42df-b38f-1b28d893589b None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.274541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 724/1389] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:43.282063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54436c-fe65-4e92-9005-7a5eaeefe33a None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.282777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54436c-fe65-4e92-9005-7a5eaeefe33a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.284714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54436c-fe65-4e92-9005-7a5eaeefe33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.418636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0b54436c-fe65-4e92-9005-7a5eaeefe33a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:43.420095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0b54436c-fe65-4e92-9005-7a5eaeefe33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:43.555785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 666/1390] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:43.618077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb330f8c-8c84-4952-be68-bb34bcc08ea7 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.618369 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb330f8c-8c84-4952-be68-bb34bcc08ea7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.618646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb330f8c-8c84-4952-be68-bb34bcc08ea7 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.620033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 725/1391] 10.222.0.70 () {54 vars in 966 bytes} [Tue Jul 19 20:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:43.628831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384f3a8-9cf2-45ed-bd6c-276f17825775 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.629122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384f3a8-9cf2-45ed-bd6c-276f17825775 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.629437 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384f3a8-9cf2-45ed-bd6c-276f17825775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.668451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b50bf-2d2f-4cbb-9264-3f42aebd272c None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.668731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b50bf-2d2f-4cbb-9264-3f42aebd272c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.669002 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b50bf-2d2f-4cbb-9264-3f42aebd272c None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.670428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 726/1392] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:43.676109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9d920-9be2-4a8d-8284-f6e943c242ea None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.676435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9d920-9be2-4a8d-8284-f6e943c242ea None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.676702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9d920-9be2-4a8d-8284-f6e943c242ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.723276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-6384f3a8-9cf2-45ed-bd6c-276f17825775 None None] MFA Rules not processed for user `82e1192b2d554199a08debb4aa392479`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:43.725129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6384f3a8-9cf2-45ed-bd6c-276f17825775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:43.742756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:43.758486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-71d9d920-9be2-4a8d-8284-f6e943c242ea None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:43.759861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-71d9d920-9be2-4a8d-8284-f6e943c242ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:43.828297 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 667/1393] 10.222.0.70 () {58 vars in 1063 bytes} [Tue Jul 19 20:41:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:43.836814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384f3a8-9cf2-45ed-bd6c-276f17825775 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:43.846643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384f3a8-9cf2-45ed-bd6c-276f17825775 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:43.873160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 727/1394] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:41:43.920335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9d920-9be2-4a8d-8284-f6e943c242ea None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:43.921631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-71d9d920-9be2-4a8d-8284-f6e943c242ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:43.937930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:43.950999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73676b07-85d2-4204-b245-7762f60d7ab0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82e1192b2d554199a08debb4aa392479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:43.951635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73676b07-85d2-4204-b245-7762f60d7ab0 service glance] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.951921 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73676b07-85d2-4204-b245-7762f60d7ab0 service glance] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.952227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73676b07-85d2-4204-b245-7762f60d7ab0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.953989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-73676b07-85d2-4204-b245-7762f60d7ab0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:44.074376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9d920-9be2-4a8d-8284-f6e943c242ea None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:44.075863 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:44.130190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73676b07-85d2-4204-b245-7762f60d7ab0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:44.132545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73676b07-85d2-4204-b245-7762f60d7ab0 service glance] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:44.164553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 668/1395] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:41:44.231727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d41155-ebf8-4032-b07f-08568cfd42c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:44.232444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d41155-ebf8-4032-b07f-08568cfd42c2 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:44.232733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d41155-ebf8-4032-b07f-08568cfd42c2 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:44.233009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d41155-ebf8-4032-b07f-08568cfd42c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:44.234874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-60d41155-ebf8-4032-b07f-08568cfd42c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:44.237280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:44.331513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d41155-ebf8-4032-b07f-08568cfd42c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:44.333788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d41155-ebf8-4032-b07f-08568cfd42c2 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:44.354342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 728/1396] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:41:44.950171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc9ffe-f409-406a-aa25-066a09cee01b None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:44.950964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc9ffe-f409-406a-aa25-066a09cee01b None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:44.951127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc9ffe-f409-406a-aa25-066a09cee01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.028476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a6bc9ffe-f409-406a-aa25-066a09cee01b None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:45.030084 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a6bc9ffe-f409-406a-aa25-066a09cee01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:45.148036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 669/1397] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:41:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:45.199034 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4ba69-b76d-4db8-9030-cf310bb22b9a None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.199416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4ba69-b76d-4db8-9030-cf310bb22b9a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.199800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4ba69-b76d-4db8-9030-cf310bb22b9a None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.201289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 729/1398] 10.222.0.70 () {54 vars in 943 bytes} [Tue Jul 19 20:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:45.208954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23208450-9afc-4c8a-b321-22b97653752f None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.209243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23208450-9afc-4c8a-b321-22b97653752f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.209557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23208450-9afc-4c8a-b321-22b97653752f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.281003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:45.297670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-23208450-9afc-4c8a-b321-22b97653752f None None] MFA Rules not processed for user `334a380fa68240c79a72746ee4a5f9dc`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:45.298911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-23208450-9afc-4c8a-b321-22b97653752f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:45.450808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 670/1399] 10.222.0.70 () {58 vars in 1040 bytes} [Tue Jul 19 20:41:45 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:45.463054 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4ba69-b76d-4db8-9030-cf310bb22b9a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:45.464586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0af4ba69-b76d-4db8-9030-cf310bb22b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:45.610736 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:45.619206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4ba69-b76d-4db8-9030-cf310bb22b9a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:45.732480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20f145-3db5-4008-b27e-8c8c199f31ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:45.733040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20f145-3db5-4008-b27e-8c8c199f31ac service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.733326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20f145-3db5-4008-b27e-8c8c199f31ac service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.733685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20f145-3db5-4008-b27e-8c8c199f31ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.735372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ef20f145-3db5-4008-b27e-8c8c199f31ac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:45.822188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20f145-3db5-4008-b27e-8c8c199f31ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:45.823818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20f145-3db5-4008-b27e-8c8c199f31ac service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:45.832479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 730/1400] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:41:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:41:46.602071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d881a9b-1884-42a4-a00d-1536ce043987 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:46.602650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d881a9b-1884-42a4-a00d-1536ce043987 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:46.602650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d881a9b-1884-42a4-a00d-1536ce043987 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:46.604391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 671/1401] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:46.609903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fa77a-c097-40cc-9fd8-d2b05e2555f9 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:46.610404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fa77a-c097-40cc-9fd8-d2b05e2555f9 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:46.610525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fa77a-c097-40cc-9fd8-d2b05e2555f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:46.642487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:46.683135 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-585fa77a-c097-40cc-9fd8-d2b05e2555f9 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:46.684394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-585fa77a-c097-40cc-9fd8-d2b05e2555f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:46.794773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 731/1402] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:41:46.840429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d881a9b-1884-42a4-a00d-1536ce043987 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:46.841989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4d881a9b-1884-42a4-a00d-1536ce043987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:46.965741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:47.018261 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d881a9b-1884-42a4-a00d-1536ce043987 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:47.171168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c1a85-ce27-4f58-8325-b1788b188791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:47.171831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c1a85-ce27-4f58-8325-b1788b188791 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:47.172117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c1a85-ce27-4f58-8325-b1788b188791 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:47.172397 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c1a85-ce27-4f58-8325-b1788b188791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:47.174135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-701c1a85-ce27-4f58-8325-b1788b188791 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:47.256708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:47.276568 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c1a85-ce27-4f58-8325-b1788b188791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:47.278477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c1a85-ce27-4f58-8325-b1788b188791 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:47.299620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 672/1403] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:41:49.607942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efddb5-4a48-4153-99c4-d555bd819d15 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:49.608536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efddb5-4a48-4153-99c4-d555bd819d15 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:49.608536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efddb5-4a48-4153-99c4-d555bd819d15 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:49.613736 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 732/1404] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:49.615880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986c930-5417-4946-bd2e-f14098812e9d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:49.616341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986c930-5417-4946-bd2e-f14098812e9d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:49.616430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986c930-5417-4946-bd2e-f14098812e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:49.690811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-3986c930-5417-4946-bd2e-f14098812e9d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:49.692060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3986c930-5417-4946-bd2e-f14098812e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:49.806871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 673/1405] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:49.856740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efddb5-4a48-4153-99c4-d555bd819d15 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:49.858651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d7efddb5-4a48-4153-99c4-d555bd819d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:49.994297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:50.076719 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efddb5-4a48-4153-99c4-d555bd819d15 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:50.226195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f5a52-d3fd-4de4-9aa9-6f5df8460615 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:50.227084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f5a52-d3fd-4de4-9aa9-6f5df8460615 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:50.227503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f5a52-d3fd-4de4-9aa9-6f5df8460615 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:50.227885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f5a52-d3fd-4de4-9aa9-6f5df8460615 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:50.229732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ab8f5a52-d3fd-4de4-9aa9-6f5df8460615 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:50.361081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:50.396240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f5a52-d3fd-4de4-9aa9-6f5df8460615 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:50.398283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f5a52-d3fd-4de4-9aa9-6f5df8460615 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:50.420133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 733/1406] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:41:52.698236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce869-ebc4-4299-a391-86c0a35ecfbc None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:52.698773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce869-ebc4-4299-a391-86c0a35ecfbc None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:52.698773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce869-ebc4-4299-a391-86c0a35ecfbc None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:52.700129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 674/1407] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:52.705940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cba64-bbd8-41c0-a0dc-aeea6756202e None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:52.706744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cba64-bbd8-41c0-a0dc-aeea6756202e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:52.706744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cba64-bbd8-41c0-a0dc-aeea6756202e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:52.786881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0f4cba64-bbd8-41c0-a0dc-aeea6756202e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:52.788258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0f4cba64-bbd8-41c0-a0dc-aeea6756202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:52.910918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 734/1408] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:41:52.965482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ce869-ebc4-4299-a391-86c0a35ecfbc None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:52.967307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-699ce869-ebc4-4299-a391-86c0a35ecfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:53.062423 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:53.233865 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ce869-ebc4-4299-a391-86c0a35ecfbc None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:53.444565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d32f04-5e16-4510-97b4-5160dc6e62f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:53.445331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d32f04-5e16-4510-97b4-5160dc6e62f0 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:53.445646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d32f04-5e16-4510-97b4-5160dc6e62f0 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:53.445935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d32f04-5e16-4510-97b4-5160dc6e62f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:53.447662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-50d32f04-5e16-4510-97b4-5160dc6e62f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:53.469694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:53.607952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d32f04-5e16-4510-97b4-5160dc6e62f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:53.609753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d32f04-5e16-4510-97b4-5160dc6e62f0 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:53.645445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 675/1409] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:42:02.908589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=81 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:03.642019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=40 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:09.090377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520030f1-de1c-4416-965e-5d7f8606334b None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:09.090955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520030f1-de1c-4416-965e-5d7f8606334b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:09.090955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520030f1-de1c-4416-965e-5d7f8606334b None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:09.092316 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 735/1410] 10.222.0.70 () {54 vars in 918 bytes} [Tue Jul 19 20:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:09.097694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863549b-d341-44cd-8e3a-cbbe12bf7562 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:09.098144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863549b-d341-44cd-8e3a-cbbe12bf7562 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:09.098281 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863549b-d341-44cd-8e3a-cbbe12bf7562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:09.174226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a863549b-d341-44cd-8e3a-cbbe12bf7562 None None] MFA Rules not processed for user `f27871b39b9b4edaa512bd5b7467174a`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:09.175493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a863549b-d341-44cd-8e3a-cbbe12bf7562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:09.288764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 676/1411] 10.222.0.70 () {58 vars in 1015 bytes} [Tue Jul 19 20:42:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:09.303842 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccf8d3-4854-478f-8ef4-9b994dbd3ef9 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:09.304130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccf8d3-4854-478f-8ef4-9b994dbd3ef9 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:09.304409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccf8d3-4854-478f-8ef4-9b994dbd3ef9 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:09.305833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 736/1412] 10.222.0.70 () {54 vars in 918 bytes} [Tue Jul 19 20:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:09.310730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3da0f0-904f-4359-8f99-8383268e14c1 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:09.311024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3da0f0-904f-4359-8f99-8383268e14c1 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:09.311308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3da0f0-904f-4359-8f99-8383268e14c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:09.386321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:09.389256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-6a3da0f0-904f-4359-8f99-8383268e14c1 None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:09.391230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6a3da0f0-904f-4359-8f99-8383268e14c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:09.510860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 677/1413] 10.222.0.70 () {58 vars in 1015 bytes} [Tue Jul 19 20:42:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:12.165081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7097ff-5f2f-4e21-885e-c921a266f8bc None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:12.165703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7097ff-5f2f-4e21-885e-c921a266f8bc None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:12.165703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7097ff-5f2f-4e21-885e-c921a266f8bc None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:12.167171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 737/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:12.172814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cb362-a55a-4aa7-a6d9-c38a1bad75b4 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:12.173298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cb362-a55a-4aa7-a6d9-c38a1bad75b4 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:12.173497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cb362-a55a-4aa7-a6d9-c38a1bad75b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:12.241389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-bf2cb362-a55a-4aa7-a6d9-c38a1bad75b4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:12.242539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bf2cb362-a55a-4aa7-a6d9-c38a1bad75b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:12.300690 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:12.347400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 678/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:12.394578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7097ff-5f2f-4e21-885e-c921a266f8bc None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:12.396181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3e7097ff-5f2f-4e21-885e-c921a266f8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:12.554368 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7097ff-5f2f-4e21-885e-c921a266f8bc None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:12.631717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:12.674435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0ad5f-f867-4213-9353-4c3d17a3a59f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:12.675089 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0ad5f-f867-4213-9353-4c3d17a3a59f service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:12.675376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0ad5f-f867-4213-9353-4c3d17a3a59f service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:12.675657 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0ad5f-f867-4213-9353-4c3d17a3a59f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:12.677942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d1b0ad5f-f867-4213-9353-4c3d17a3a59f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:12.774329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0ad5f-f867-4213-9353-4c3d17a3a59f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:12.776582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0ad5f-f867-4213-9353-4c3d17a3a59f service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:12.796126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 738/1416] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:13.744134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741c12a-4c9a-422f-9a46-d869dc15328a None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:13.744903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741c12a-4c9a-422f-9a46-d869dc15328a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:13.744903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741c12a-4c9a-422f-9a46-d869dc15328a None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:13.746505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 679/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:42:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:13.751979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95ffba-88ed-4327-9b4a-fa0fbf358e72 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:13.752648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95ffba-88ed-4327-9b4a-fa0fbf358e72 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:13.752648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95ffba-88ed-4327-9b4a-fa0fbf358e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:13.847928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-5e95ffba-88ed-4327-9b4a-fa0fbf358e72 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:13.849472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5e95ffba-88ed-4327-9b4a-fa0fbf358e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:13.885670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:13.971713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 739/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:14.121413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741c12a-4c9a-422f-9a46-d869dc15328a None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:14.122978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8741c12a-4c9a-422f-9a46-d869dc15328a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:14.305598 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741c12a-4c9a-422f-9a46-d869dc15328a None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:14.350617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:14.430183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c0c65-3f2a-4bf6-b032-8e8e85d74770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:14.430838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0c65-3f2a-4bf6-b032-8e8e85d74770 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:14.431121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0c65-3f2a-4bf6-b032-8e8e85d74770 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:14.431448 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0c65-3f2a-4bf6-b032-8e8e85d74770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:14.433071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3c8c0c65-3f2a-4bf6-b032-8e8e85d74770 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:14.531875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c0c65-3f2a-4bf6-b032-8e8e85d74770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:14.533419 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c0c65-3f2a-4bf6-b032-8e8e85d74770 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:14.554859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 680/1419] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:15.455312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b44b-d89b-473a-8b5f-4362a4640375 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:15.455865 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b44b-d89b-473a-8b5f-4362a4640375 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:15.456019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3b44b-d89b-473a-8b5f-4362a4640375 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:15.458163 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 740/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:42:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:15.463364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3429bb-d8fa-4de1-aa59-23d27bd4b86f None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:15.463817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3429bb-d8fa-4de1-aa59-23d27bd4b86f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:15.464042 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3429bb-d8fa-4de1-aa59-23d27bd4b86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:15.519252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:15.543581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0c3429bb-d8fa-4de1-aa59-23d27bd4b86f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:15.544822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0c3429bb-d8fa-4de1-aa59-23d27bd4b86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:15.640938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 681/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:15.795633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3b44b-d89b-473a-8b5f-4362a4640375 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:15.797218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c5a3b44b-d89b-473a-8b5f-4362a4640375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:15.951713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3b44b-d89b-473a-8b5f-4362a4640375 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:15.968363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:16.077776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa635706-dfcf-45eb-b8f6-3fccd5310e5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:16.078429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa635706-dfcf-45eb-b8f6-3fccd5310e5c service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:16.078710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa635706-dfcf-45eb-b8f6-3fccd5310e5c service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:16.078988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa635706-dfcf-45eb-b8f6-3fccd5310e5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:16.080650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-aa635706-dfcf-45eb-b8f6-3fccd5310e5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:16.183075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa635706-dfcf-45eb-b8f6-3fccd5310e5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:16.184600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa635706-dfcf-45eb-b8f6-3fccd5310e5c service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:16.203346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 741/1422] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:42:17.100039 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427e2ae-2daf-42a0-b5aa-f4dbd6725895 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:17.100699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427e2ae-2daf-42a0-b5aa-f4dbd6725895 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:17.100699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427e2ae-2daf-42a0-b5aa-f4dbd6725895 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:17.102333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 682/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:17.107722 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce439695-06a1-4eff-ae5f-b0cab5c5d568 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:17.108247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce439695-06a1-4eff-ae5f-b0cab5c5d568 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:17.108371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce439695-06a1-4eff-ae5f-b0cab5c5d568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:17.127009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:17.183859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-ce439695-06a1-4eff-ae5f-b0cab5c5d568 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:17.185070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ce439695-06a1-4eff-ae5f-b0cab5c5d568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:17.283569 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 742/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:17.434015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427e2ae-2daf-42a0-b5aa-f4dbd6725895 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:17.435559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e427e2ae-2daf-42a0-b5aa-f4dbd6725895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:17.549973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:17.587168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427e2ae-2daf-42a0-b5aa-f4dbd6725895 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:17.704594 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fda68-b39b-43ac-b51b-7d9cbcd4db26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:17.705265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fda68-b39b-43ac-b51b-7d9cbcd4db26 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:17.705585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fda68-b39b-43ac-b51b-7d9cbcd4db26 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:17.705868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fda68-b39b-43ac-b51b-7d9cbcd4db26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:17.708018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3c2fda68-b39b-43ac-b51b-7d9cbcd4db26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:17.793927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:17.808544 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fda68-b39b-43ac-b51b-7d9cbcd4db26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:17.810191 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fda68-b39b-43ac-b51b-7d9cbcd4db26 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:17.827784 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 683/1425] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:18.716766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eced72-4d24-4818-aa38-f251f43b2fac None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:18.717293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eced72-4d24-4818-aa38-f251f43b2fac None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:18.717293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eced72-4d24-4818-aa38-f251f43b2fac None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:18.718646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 743/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:18.723634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d7077-de58-4fd7-9478-a55672bc3d40 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:18.724045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d7077-de58-4fd7-9478-a55672bc3d40 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:18.724197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d7077-de58-4fd7-9478-a55672bc3d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:18.801171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-b02d7077-de58-4fd7-9478-a55672bc3d40 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:18.802432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b02d7077-de58-4fd7-9478-a55672bc3d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:18.906976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 684/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:19.061834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eced72-4d24-4818-aa38-f251f43b2fac None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:19.063732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-78eced72-4d24-4818-aa38-f251f43b2fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:19.148382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:19.219566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eced72-4d24-4818-aa38-f251f43b2fac None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:19.335685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d4cb0-1a57-4610-a276-3f5f80b5cf34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:19.336260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d4cb0-1a57-4610-a276-3f5f80b5cf34 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:19.336547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d4cb0-1a57-4610-a276-3f5f80b5cf34 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:19.336823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d4cb0-1a57-4610-a276-3f5f80b5cf34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:19.338543 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f85d4cb0-1a57-4610-a276-3f5f80b5cf34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:19.390807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:19.436140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d4cb0-1a57-4610-a276-3f5f80b5cf34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:19.437827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d4cb0-1a57-4610-a276-3f5f80b5cf34 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:19.462988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 744/1428] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:20.290030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562e4b6-4807-4604-9160-54e4b6ed1ecc None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:20.290647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562e4b6-4807-4604-9160-54e4b6ed1ecc None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:20.290647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562e4b6-4807-4604-9160-54e4b6ed1ecc None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:20.292014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 685/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:20.297836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf60dd1-8b42-4b75-8780-a7a0fc4ef099 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:20.298439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf60dd1-8b42-4b75-8780-a7a0fc4ef099 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:20.298439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf60dd1-8b42-4b75-8780-a7a0fc4ef099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:20.381384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-fdf60dd1-8b42-4b75-8780-a7a0fc4ef099 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:20.382623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fdf60dd1-8b42-4b75-8780-a7a0fc4ef099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:20.483031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 745/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:20.645740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562e4b6-4807-4604-9160-54e4b6ed1ecc None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:20.647413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4562e4b6-4807-4604-9160-54e4b6ed1ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:20.704715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:20.830119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562e4b6-4807-4604-9160-54e4b6ed1ecc None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:20.955215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608da5d-3117-4626-9fba-9868f39903f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:20.955834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608da5d-3117-4626-9fba-9868f39903f7 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:20.956129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608da5d-3117-4626-9fba-9868f39903f7 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:20.956444 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608da5d-3117-4626-9fba-9868f39903f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:20.958530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d608da5d-3117-4626-9fba-9868f39903f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:20.965985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:21.058107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608da5d-3117-4626-9fba-9868f39903f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:21.059728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608da5d-3117-4626-9fba-9868f39903f7 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:21.077869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 686/1431] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:21.927730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3464237-e440-4f7c-8042-aca46d214394 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:21.928299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3464237-e440-4f7c-8042-aca46d214394 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:21.928299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3464237-e440-4f7c-8042-aca46d214394 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:21.929712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 746/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:21.936053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd284c-8ff5-41b3-a603-883ef735857c None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:21.936633 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd284c-8ff5-41b3-a603-883ef735857c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:21.936633 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd284c-8ff5-41b3-a603-883ef735857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:22.027843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-46dd284c-8ff5-41b3-a603-883ef735857c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:22.029122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-46dd284c-8ff5-41b3-a603-883ef735857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:22.107478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:22.141887 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 687/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:42:22.313501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3464237-e440-4f7c-8042-aca46d214394 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:22.315040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f3464237-e440-4f7c-8042-aca46d214394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:22.335155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:22.491115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3464237-e440-4f7c-8042-aca46d214394 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:22.586489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:22.622329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea0aa8-34ca-420b-98c5-4c476cbfa212 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:22.622987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea0aa8-34ca-420b-98c5-4c476cbfa212 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:22.623268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea0aa8-34ca-420b-98c5-4c476cbfa212 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:22.623542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea0aa8-34ca-420b-98c5-4c476cbfa212 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:22.625218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d8ea0aa8-34ca-420b-98c5-4c476cbfa212 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:22.723748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea0aa8-34ca-420b-98c5-4c476cbfa212 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:22.725396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea0aa8-34ca-420b-98c5-4c476cbfa212 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:22.745852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 747/1434] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:42:23.568416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c84bc2-6578-4ef0-9a26-4c36facc0dc9 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:23.569026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c84bc2-6578-4ef0-9a26-4c36facc0dc9 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:23.569026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c84bc2-6578-4ef0-9a26-4c36facc0dc9 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:23.570430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 688/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:23.575608 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5659030-0d3f-4a62-b8bd-92e63b086ade None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:23.576114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5659030-0d3f-4a62-b8bd-92e63b086ade None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:23.576232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5659030-0d3f-4a62-b8bd-92e63b086ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:23.661824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a5659030-0d3f-4a62-b8bd-92e63b086ade None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:23.662925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a5659030-0d3f-4a62-b8bd-92e63b086ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:23.707812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:23.768439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 748/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:23.919582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c84bc2-6578-4ef0-9a26-4c36facc0dc9 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:23.921063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-57c84bc2-6578-4ef0-9a26-4c36facc0dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:24.089313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c84bc2-6578-4ef0-9a26-4c36facc0dc9 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:24.145300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:24.205143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25fdd2-21c6-45d7-82cc-0ed2a02ff021 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:24.205727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25fdd2-21c6-45d7-82cc-0ed2a02ff021 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:24.206011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25fdd2-21c6-45d7-82cc-0ed2a02ff021 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:24.206287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25fdd2-21c6-45d7-82cc-0ed2a02ff021 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:24.208203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1c25fdd2-21c6-45d7-82cc-0ed2a02ff021 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:24.305809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25fdd2-21c6-45d7-82cc-0ed2a02ff021 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:24.307426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25fdd2-21c6-45d7-82cc-0ed2a02ff021 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:24.327248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 689/1437] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:25.156697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee85ae-c747-422d-959e-5ae71544b2a3 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:25.157290 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee85ae-c747-422d-959e-5ae71544b2a3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:25.157290 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee85ae-c747-422d-959e-5ae71544b2a3 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:25.158717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 749/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:42:25.165707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106d170-5ea1-493b-8389-ff5462026a7c None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:25.166298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106d170-5ea1-493b-8389-ff5462026a7c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:25.166636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106d170-5ea1-493b-8389-ff5462026a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:25.270354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0106d170-5ea1-493b-8389-ff5462026a7c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:25.271604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0106d170-5ea1-493b-8389-ff5462026a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:25.273862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:25.401424 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 690/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:25.621737 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee85ae-c747-422d-959e-5ae71544b2a3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:25.623856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ffee85ae-c747-422d-959e-5ae71544b2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:25.779664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee85ae-c747-422d-959e-5ae71544b2a3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:25.807354 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:25.902618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a168f17-0184-47f0-b5ce-37b60bf9b8f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:25.903272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a168f17-0184-47f0-b5ce-37b60bf9b8f7 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:25.903555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a168f17-0184-47f0-b5ce-37b60bf9b8f7 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:25.903836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a168f17-0184-47f0-b5ce-37b60bf9b8f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:25.905994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-7a168f17-0184-47f0-b5ce-37b60bf9b8f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:26.003994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a168f17-0184-47f0-b5ce-37b60bf9b8f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:26.006865 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a168f17-0184-47f0-b5ce-37b60bf9b8f7 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:26.032180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 750/1440] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:28.151259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe0473-ca24-47b9-bbbf-0c4ab6bc4350 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:28.152091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe0473-ca24-47b9-bbbf-0c4ab6bc4350 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:28.152254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe0473-ca24-47b9-bbbf-0c4ab6bc4350 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:28.154695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 691/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:42:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:28.162594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa5842-7e70-43e0-8553-4e833bf708ef None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:28.163249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa5842-7e70-43e0-8553-4e833bf708ef None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:28.163249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa5842-7e70-43e0-8553-4e833bf708ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:28.213519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:28.246573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-effa5842-7e70-43e0-8553-4e833bf708ef None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:28.247862 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-effa5842-7e70-43e0-8553-4e833bf708ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:28.380500 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 751/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:28.421530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe0473-ca24-47b9-bbbf-0c4ab6bc4350 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:28.423156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a2fe0473-ca24-47b9-bbbf-0c4ab6bc4350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:28.535004 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe0473-ca24-47b9-bbbf-0c4ab6bc4350 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:28.575490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:28.627608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f4269-c54c-4830-b36b-cf83ecbffdc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:28.628843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4269-c54c-4830-b36b-cf83ecbffdc2 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:28.628843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4269-c54c-4830-b36b-cf83ecbffdc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:28.628843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4269-c54c-4830-b36b-cf83ecbffdc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:28.630663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b68f4269-c54c-4830-b36b-cf83ecbffdc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:28.774984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f4269-c54c-4830-b36b-cf83ecbffdc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:28.776402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f4269-c54c-4830-b36b-cf83ecbffdc2 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:28.795439 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 692/1443] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:31.157042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee40e36-2c95-4457-a20d-ef681d647c37 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:31.157711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee40e36-2c95-4457-a20d-ef681d647c37 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:31.158102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee40e36-2c95-4457-a20d-ef681d647c37 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:31.161404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 752/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:42:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:31.169996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee137d70-289a-4e1f-83dd-a32e28795440 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:31.170862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee137d70-289a-4e1f-83dd-a32e28795440 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:31.171056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee137d70-289a-4e1f-83dd-a32e28795440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:31.253761 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ee137d70-289a-4e1f-83dd-a32e28795440 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:31.255610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ee137d70-289a-4e1f-83dd-a32e28795440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:31.271285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:31.372611 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 693/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:31.414510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee40e36-2c95-4457-a20d-ef681d647c37 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:31.416026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8ee40e36-2c95-4457-a20d-ef681d647c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:31.564064 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee40e36-2c95-4457-a20d-ef681d647c37 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:31.583125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:31.674067 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f8ee5-5206-441c-bd80-b8233c1f4154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:31.675094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f8ee5-5206-441c-bd80-b8233c1f4154 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:31.675587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f8ee5-5206-441c-bd80-b8233c1f4154 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:31.676056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f8ee5-5206-441c-bd80-b8233c1f4154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:31.679109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a29f8ee5-5206-441c-bd80-b8233c1f4154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:31.870185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f8ee5-5206-441c-bd80-b8233c1f4154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:31.871663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f8ee5-5206-441c-bd80-b8233c1f4154 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:31.909492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 753/1446] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:35.105920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2ca41-5992-47b8-b911-92da54c5df9e None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:35.106556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2ca41-5992-47b8-b911-92da54c5df9e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:35.106556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2ca41-5992-47b8-b911-92da54c5df9e None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:35.107879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 694/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:35.114991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39353224-e74e-49fd-9557-26a91de8a040 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:35.115605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39353224-e74e-49fd-9557-26a91de8a040 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:35.115974 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39353224-e74e-49fd-9557-26a91de8a040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:35.201952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:35.239003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-39353224-e74e-49fd-9557-26a91de8a040 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:35.241089 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-39353224-e74e-49fd-9557-26a91de8a040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:35.391174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 754/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:35.460562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2ca41-5992-47b8-b911-92da54c5df9e None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:35.462240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6bb2ca41-5992-47b8-b911-92da54c5df9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:35.583999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2ca41-5992-47b8-b911-92da54c5df9e None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:35.641330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:35.697864 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cb70a-8e5e-473b-8c7f-d6ff0723cb50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:35.698623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cb70a-8e5e-473b-8c7f-d6ff0723cb50 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:35.698928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cb70a-8e5e-473b-8c7f-d6ff0723cb50 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:35.699341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cb70a-8e5e-473b-8c7f-d6ff0723cb50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:35.702272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-eb6cb70a-8e5e-473b-8c7f-d6ff0723cb50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:35.866311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cb70a-8e5e-473b-8c7f-d6ff0723cb50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:35.867871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cb70a-8e5e-473b-8c7f-d6ff0723cb50 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:35.889334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 695/1449] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:42:38.936884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eaf550-f41f-4420-b558-348bb6fd8b6f None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:38.937801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eaf550-f41f-4420-b558-348bb6fd8b6f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:38.937801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eaf550-f41f-4420-b558-348bb6fd8b6f None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:38.938886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 755/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:42:38.945994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61751d-6d55-4c13-93d5-07795be5b35e None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:38.946843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61751d-6d55-4c13-93d5-07795be5b35e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:38.947061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61751d-6d55-4c13-93d5-07795be5b35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:39.043771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ae61751d-6d55-4c13-93d5-07795be5b35e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:39.045007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ae61751d-6d55-4c13-93d5-07795be5b35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:39.071484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:39.175768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 696/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:39.218563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eaf550-f41f-4420-b558-348bb6fd8b6f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:39.221028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-92eaf550-f41f-4420-b558-348bb6fd8b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:39.353970 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eaf550-f41f-4420-b558-348bb6fd8b6f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:39.381127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:39.449507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a55d9e-cfdc-4e4a-b9e8-1f3893bba5e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:39.450264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a55d9e-cfdc-4e4a-b9e8-1f3893bba5e1 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:39.450601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a55d9e-cfdc-4e4a-b9e8-1f3893bba5e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:39.450995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a55d9e-cfdc-4e4a-b9e8-1f3893bba5e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:39.452848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-79a55d9e-cfdc-4e4a-b9e8-1f3893bba5e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:39.610597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a55d9e-cfdc-4e4a-b9e8-1f3893bba5e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:39.612039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a55d9e-cfdc-4e4a-b9e8-1f3893bba5e1 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:39.634798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 756/1452] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:42:41.930335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bda79f-5ba9-4111-874d-68fc7c57095f None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:41.930886 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bda79f-5ba9-4111-874d-68fc7c57095f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:41.930886 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bda79f-5ba9-4111-874d-68fc7c57095f None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:41.932293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 697/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:41.938305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f6ca7-61a5-4a79-a75b-459a77f50edd None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:41.938782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f6ca7-61a5-4a79-a75b-459a77f50edd None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:41.939037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f6ca7-61a5-4a79-a75b-459a77f50edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:42.012320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:42.016256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-cd1f6ca7-61a5-4a79-a75b-459a77f50edd None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:42.018339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-cd1f6ca7-61a5-4a79-a75b-459a77f50edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:42.124879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 757/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:42.163880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bda79f-5ba9-4111-874d-68fc7c57095f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:42.165500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d2bda79f-5ba9-4111-874d-68fc7c57095f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:42.281780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bda79f-5ba9-4111-874d-68fc7c57095f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:42.334294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:42.360824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790b0b7-4ed4-4992-a1f2-70374cf679f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:42.361518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790b0b7-4ed4-4992-a1f2-70374cf679f4 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:42.361819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790b0b7-4ed4-4992-a1f2-70374cf679f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:42.362092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790b0b7-4ed4-4992-a1f2-70374cf679f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:42.364234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a790b0b7-4ed4-4992-a1f2-70374cf679f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:42.481517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790b0b7-4ed4-4992-a1f2-70374cf679f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:42.483128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790b0b7-4ed4-4992-a1f2-70374cf679f4 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:42.508798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 698/1455] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:44.718329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743d165-9486-446a-86c0-d63182cad5ee None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:44.718882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743d165-9486-446a-86c0-d63182cad5ee None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:44.718882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743d165-9486-446a-86c0-d63182cad5ee None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:44.720180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 758/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:44.726219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37229087-d716-4f76-971f-213378bf2233 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:44.726742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37229087-d716-4f76-971f-213378bf2233 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:44.726841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37229087-d716-4f76-971f-213378bf2233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:44.801627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-37229087-d716-4f76-971f-213378bf2233 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:44.802879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-37229087-d716-4f76-971f-213378bf2233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:44.837114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:44.921473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 699/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:44.964175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743d165-9486-446a-86c0-d63182cad5ee None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:44.966100 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6743d165-9486-446a-86c0-d63182cad5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:45.102376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743d165-9486-446a-86c0-d63182cad5ee None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:45.138764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:45.197380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5ee97-15e4-4713-b149-7b5440df6f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:45.198060 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5ee97-15e4-4713-b149-7b5440df6f80 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:45.198346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5ee97-15e4-4713-b149-7b5440df6f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:45.198625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5ee97-15e4-4713-b149-7b5440df6f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:45.200330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-04b5ee97-15e4-4713-b149-7b5440df6f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:45.324334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5ee97-15e4-4713-b149-7b5440df6f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:45.325936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5ee97-15e4-4713-b149-7b5440df6f80 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:45.344648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 759/1458] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:47.485086 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb64272-01f8-4480-96ab-e6f63dcb7ba5 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:47.485687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb64272-01f8-4480-96ab-e6f63dcb7ba5 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:47.485687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb64272-01f8-4480-96ab-e6f63dcb7ba5 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:47.487111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 700/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:47.492942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb643a-2e6f-449f-bddf-df9282cc7288 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:47.493462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb643a-2e6f-449f-bddf-df9282cc7288 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:47.493560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb643a-2e6f-449f-bddf-df9282cc7288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:47.564434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f9eb643a-2e6f-449f-bddf-df9282cc7288 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:47.565750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f9eb643a-2e6f-449f-bddf-df9282cc7288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:47.574367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:47.670260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 760/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:47.719606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb64272-01f8-4480-96ab-e6f63dcb7ba5 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:47.721501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-fdb64272-01f8-4480-96ab-e6f63dcb7ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:47.841177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb64272-01f8-4480-96ab-e6f63dcb7ba5 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:48.021775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:48.044489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acd964-5a66-4701-9321-2adefb32b9be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:48.045291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acd964-5a66-4701-9321-2adefb32b9be service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:48.045713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acd964-5a66-4701-9321-2adefb32b9be service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:48.046093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acd964-5a66-4701-9321-2adefb32b9be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:48.048059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-24acd964-5a66-4701-9321-2adefb32b9be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:48.171701 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24acd964-5a66-4701-9321-2adefb32b9be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:48.173133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24acd964-5a66-4701-9321-2adefb32b9be service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:48.197125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 701/1461] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:50.360115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ac759-20c0-48fb-823e-d70f3e2ce5c8 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:50.360995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ac759-20c0-48fb-823e-d70f3e2ce5c8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:50.360995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ac759-20c0-48fb-823e-d70f3e2ce5c8 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:50.362477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 761/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:42:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:42:50.369095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4220ab-03f9-4439-9cf1-b12b3bd9b7b6 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:50.369711 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4220ab-03f9-4439-9cf1-b12b3bd9b7b6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:50.369816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4220ab-03f9-4439-9cf1-b12b3bd9b7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:50.461295 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-4a4220ab-03f9-4439-9cf1-b12b3bd9b7b6 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:50.462565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4a4220ab-03f9-4439-9cf1-b12b3bd9b7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:50.508655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:50.579139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 702/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:50.618152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ac759-20c0-48fb-823e-d70f3e2ce5c8 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:50.619805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-776ac759-20c0-48fb-823e-d70f3e2ce5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:50.753177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ac759-20c0-48fb-823e-d70f3e2ce5c8 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:50.795611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:50.842804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72344619-880a-4c5a-9c90-76a3d694e764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:50.843533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72344619-880a-4c5a-9c90-76a3d694e764 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:50.843920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72344619-880a-4c5a-9c90-76a3d694e764 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:50.844269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72344619-880a-4c5a-9c90-76a3d694e764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:50.846191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-72344619-880a-4c5a-9c90-76a3d694e764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:50.987625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72344619-880a-4c5a-9c90-76a3d694e764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:50.987625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72344619-880a-4c5a-9c90-76a3d694e764 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:51.010189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 762/1464] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:42:53.183938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738aa9-a758-43fa-95d2-9c4bf2ee2314 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:53.184625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738aa9-a758-43fa-95d2-9c4bf2ee2314 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:53.184625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738aa9-a758-43fa-95d2-9c4bf2ee2314 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:53.186124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 703/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:53.193341 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e049e-166c-4f82-9870-6e8b01600530 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:53.193979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e049e-166c-4f82-9870-6e8b01600530 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:53.194148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e049e-166c-4f82-9870-6e8b01600530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:53.272330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e24e049e-166c-4f82-9870-6e8b01600530 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:53.273670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e24e049e-166c-4f82-9870-6e8b01600530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:53.295139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:53.390136 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 763/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:53.433734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c738aa9-a758-43fa-95d2-9c4bf2ee2314 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:53.435573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8c738aa9-a758-43fa-95d2-9c4bf2ee2314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:53.550121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c738aa9-a758-43fa-95d2-9c4bf2ee2314 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:53.639187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:53.649278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cd30a-b2f8-4bd7-aea7-69ee533e8932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:53.649989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cd30a-b2f8-4bd7-aea7-69ee533e8932 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:53.650304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cd30a-b2f8-4bd7-aea7-69ee533e8932 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:53.650570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cd30a-b2f8-4bd7-aea7-69ee533e8932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:53.652513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-191cd30a-b2f8-4bd7-aea7-69ee533e8932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:53.811220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cd30a-b2f8-4bd7-aea7-69ee533e8932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:53.813584 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cd30a-b2f8-4bd7-aea7-69ee533e8932 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:53.837083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 704/1467] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:42:55.995390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451313c-e877-47a6-b762-93d4775c28a7 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:55.995992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451313c-e877-47a6-b762-93d4775c28a7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:55.996163 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451313c-e877-47a6-b762-93d4775c28a7 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:55.997546 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 764/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:56.005191 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee79e2-c79d-45ae-a078-54cac95c7211 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:56.005821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee79e2-c79d-45ae-a078-54cac95c7211 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:56.005821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee79e2-c79d-45ae-a078-54cac95c7211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:56.076305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ddee79e2-c79d-45ae-a078-54cac95c7211 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:56.077681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ddee79e2-c79d-45ae-a078-54cac95c7211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:56.130009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:56.180203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 705/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:42:56.219995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451313c-e877-47a6-b762-93d4775c28a7 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:56.221642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c451313c-e877-47a6-b762-93d4775c28a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:56.342083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451313c-e877-47a6-b762-93d4775c28a7 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:56.396718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:56.427550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d4aa8-df59-40b5-b2aa-d1ed000b75b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:56.428376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d4aa8-df59-40b5-b2aa-d1ed000b75b6 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:56.428704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d4aa8-df59-40b5-b2aa-d1ed000b75b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:56.429026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d4aa8-df59-40b5-b2aa-d1ed000b75b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:56.430938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-7c6d4aa8-df59-40b5-b2aa-d1ed000b75b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:56.562902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d4aa8-df59-40b5-b2aa-d1ed000b75b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:56.564371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d4aa8-df59-40b5-b2aa-d1ed000b75b6 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:56.586830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 765/1470] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:42:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:43:06.178300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=22 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:06.583928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=66 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:17.633463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc9940-6a5e-4420-addb-e6bda08a6e8e None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:17.634083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc9940-6a5e-4420-addb-e6bda08a6e8e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:17.638545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc9940-6a5e-4420-addb-e6bda08a6e8e None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:17.638545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 706/1471] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:43:17.642533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6113d7-af23-4751-9765-e6b954a80759 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:17.643105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6113d7-af23-4751-9765-e6b954a80759 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:17.643105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6113d7-af23-4751-9765-e6b954a80759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:17.721102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-1a6113d7-af23-4751-9765-e6b954a80759 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:17.722962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1a6113d7-af23-4751-9765-e6b954a80759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:17.849078 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 766/1472] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:43:17.942530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597e6a5-a8b0-4b85-b4b3-f25bb64b3cb9 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:17.942834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597e6a5-a8b0-4b85-b4b3-f25bb64b3cb9 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:17.943084 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597e6a5-a8b0-4b85-b4b3-f25bb64b3cb9 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:17.944510 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 707/1473] 10.222.0.70 () {54 vars in 966 bytes} [Tue Jul 19 20:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:43:17.950180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ceb34-79b1-4190-961a-a81fd19cd841 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:17.950608 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ceb34-79b1-4190-961a-a81fd19cd841 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:17.950975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ceb34-79b1-4190-961a-a81fd19cd841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:18.063909 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-d18ceb34-79b1-4190-961a-a81fd19cd841 None None] MFA Rules not processed for user `82e1192b2d554199a08debb4aa392479`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:18.065580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d18ceb34-79b1-4190-961a-a81fd19cd841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:18.100204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:18.182686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 767/1474] 10.222.0.70 () {58 vars in 1063 bytes} [Tue Jul 19 20:43:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:43:18.192213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597e6a5-a8b0-4b85-b4b3-f25bb64b3cb9 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:18.193853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-9597e6a5-a8b0-4b85-b4b3-f25bb64b3cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:18.310737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597e6a5-a8b0-4b85-b4b3-f25bb64b3cb9 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:18.392702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38487aa7-258b-4226-8054-3e14ad99ecf6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82e1192b2d554199a08debb4aa392479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:18.393385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38487aa7-258b-4226-8054-3e14ad99ecf6 service glance] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:18.393670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38487aa7-258b-4226-8054-3e14ad99ecf6 service glance] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:18.394026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38487aa7-258b-4226-8054-3e14ad99ecf6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:18.395689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-38487aa7-258b-4226-8054-3e14ad99ecf6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:18.455313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:18.515552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38487aa7-258b-4226-8054-3e14ad99ecf6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:18.516978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38487aa7-258b-4226-8054-3e14ad99ecf6 service glance] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:18.536982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 708/1475] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:43:20.763727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678f1d-56f3-41c7-8be7-eb01c89cbf1c None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:20.764893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678f1d-56f3-41c7-8be7-eb01c89cbf1c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:20.764893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678f1d-56f3-41c7-8be7-eb01c89cbf1c None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:20.765688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 768/1476] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:43:20.777034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87913db2-b3d2-42da-a840-69f1b3775afa None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:20.777656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87913db2-b3d2-42da-a840-69f1b3775afa None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:20.777656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87913db2-b3d2-42da-a840-69f1b3775afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:20.868858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-87913db2-b3d2-42da-a840-69f1b3775afa None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:20.870265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-87913db2-b3d2-42da-a840-69f1b3775afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:21.014291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 709/1477] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:43:21.039190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05678f1d-56f3-41c7-8be7-eb01c89cbf1c None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:21.040767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-05678f1d-56f3-41c7-8be7-eb01c89cbf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:21.185470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05678f1d-56f3-41c7-8be7-eb01c89cbf1c None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:21.246706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:21.297437 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb61e1c-90c7-4672-91ba-b5aa598ffc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:21.298119 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb61e1c-90c7-4672-91ba-b5aa598ffc73 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:21.298404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb61e1c-90c7-4672-91ba-b5aa598ffc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:21.298690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb61e1c-90c7-4672-91ba-b5aa598ffc73 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:21.299441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb61e1c-90c7-4672-91ba-b5aa598ffc73 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:21.303786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: WARNING py.warnings [None req-7fb61e1c-90c7-4672-91ba-b5aa598ffc73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:43:21.303786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: warnings.warn(msg) Jul 19 20:43:21.303786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: Jul 19 20:43:21.307080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb61e1c-90c7-4672-91ba-b5aa598ffc73 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:21.337374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 769/1478] 10.222.0.70 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:21 2022] GET /identity/v3/projects => generated 2553 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:43:23.473143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f0a4b-ca8b-47ca-a0ef-2ca9de696a47 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:23.473739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f0a4b-ca8b-47ca-a0ef-2ca9de696a47 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:23.473739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f0a4b-ca8b-47ca-a0ef-2ca9de696a47 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:23.475126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 710/1479] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:43:23.481523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec9583-475b-40aa-ab4a-0bb9bfeb07d3 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:23.482092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec9583-475b-40aa-ab4a-0bb9bfeb07d3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:23.482092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec9583-475b-40aa-ab4a-0bb9bfeb07d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:23.568402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-72ec9583-475b-40aa-ab4a-0bb9bfeb07d3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:23.569776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-72ec9583-475b-40aa-ab4a-0bb9bfeb07d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:23.681763 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 770/1480] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:43:23.729215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f0a4b-ca8b-47ca-a0ef-2ca9de696a47 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:23.731030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-130f0a4b-ca8b-47ca-a0ef-2ca9de696a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:23.742074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:23.912524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f0a4b-ca8b-47ca-a0ef-2ca9de696a47 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:24.004581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:24.045891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174580a-589f-4936-9c2a-e07d36f1cba7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:24.046540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174580a-589f-4936-9c2a-e07d36f1cba7 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:24.046822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174580a-589f-4936-9c2a-e07d36f1cba7 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:24.047094 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174580a-589f-4936-9c2a-e07d36f1cba7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:24.048853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8174580a-589f-4936-9c2a-e07d36f1cba7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:24.150517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174580a-589f-4936-9c2a-e07d36f1cba7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:24.152127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174580a-589f-4936-9c2a-e07d36f1cba7 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:24.177162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 711/1481] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:43:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:43:26.672200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31764766-c738-401d-a21b-ed70476529c3 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:26.672793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31764766-c738-401d-a21b-ed70476529c3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:26.672793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31764766-c738-401d-a21b-ed70476529c3 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:26.674111 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 771/1482] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:43:26.683499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f9399-f9d3-4a63-9032-6308c81646a6 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:26.684120 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f9399-f9d3-4a63-9032-6308c81646a6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:26.684531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f9399-f9d3-4a63-9032-6308c81646a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:26.795437 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-d99f9399-f9d3-4a63-9032-6308c81646a6 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:26.795437 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d99f9399-f9d3-4a63-9032-6308c81646a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:26.858463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:26.918289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 712/1483] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:43:26.959243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31764766-c738-401d-a21b-ed70476529c3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:26.962209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-31764766-c738-401d-a21b-ed70476529c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:27.007720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:27.100863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31764766-c738-401d-a21b-ed70476529c3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:27.190448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5fe492-e2bf-4d20-acc4-748116142846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:27.191665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fe492-e2bf-4d20-acc4-748116142846 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:27.191946 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fe492-e2bf-4d20-acc4-748116142846 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:27.192222 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fe492-e2bf-4d20-acc4-748116142846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:27.194138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-cc5fe492-e2bf-4d20-acc4-748116142846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:27.284101 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:27.324407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5fe492-e2bf-4d20-acc4-748116142846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:27.325887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5fe492-e2bf-4d20-acc4-748116142846 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:27.346045 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 772/1484] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:43:29.540827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab32cb7-6dbc-4884-9ce7-da5c8bff53e3 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:29.541773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab32cb7-6dbc-4884-9ce7-da5c8bff53e3 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:29.541773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab32cb7-6dbc-4884-9ce7-da5c8bff53e3 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:29.543064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 713/1485] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:43:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:43:29.549570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff53dd-6d94-491e-822a-78bc71e59034 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:29.550160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff53dd-6d94-491e-822a-78bc71e59034 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:29.550160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff53dd-6d94-491e-822a-78bc71e59034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:29.629277 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0cff53dd-6d94-491e-822a-78bc71e59034 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:29.630451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0cff53dd-6d94-491e-822a-78bc71e59034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:29.734583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 773/1486] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:43:29.775347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab32cb7-6dbc-4884-9ce7-da5c8bff53e3 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:29.777067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8ab32cb7-6dbc-4884-9ce7-da5c8bff53e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:29.891143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab32cb7-6dbc-4884-9ce7-da5c8bff53e3 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:29.978372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:29.981219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170762bc-f7cb-48f7-bf51-0de7fe9d2d8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:29.981962 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170762bc-f7cb-48f7-bf51-0de7fe9d2d8c service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:29.982470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170762bc-f7cb-48f7-bf51-0de7fe9d2d8c service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:29.982941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170762bc-f7cb-48f7-bf51-0de7fe9d2d8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:29.985015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-170762bc-f7cb-48f7-bf51-0de7fe9d2d8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:30.108277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170762bc-f7cb-48f7-bf51-0de7fe9d2d8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:30.109776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170762bc-f7cb-48f7-bf51-0de7fe9d2d8c service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:30.130138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 714/1487] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:43:32.379681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93159-bc61-4803-bdee-a52f09e2d954 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:32.380173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93159-bc61-4803-bdee-a52f09e2d954 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:32.380293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f93159-bc61-4803-bdee-a52f09e2d954 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:32.381685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 774/1488] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:43:32.387219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910092d-f4ab-46a6-8bee-fee016ad3845 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:32.387677 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910092d-f4ab-46a6-8bee-fee016ad3845 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:32.387783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910092d-f4ab-46a6-8bee-fee016ad3845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:32.453496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-d910092d-f4ab-46a6-8bee-fee016ad3845 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:32.454702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d910092d-f4ab-46a6-8bee-fee016ad3845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:32.509146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:32.559234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 715/1489] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:43:32.602668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f93159-bc61-4803-bdee-a52f09e2d954 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:32.604413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-37f93159-bc61-4803-bdee-a52f09e2d954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:32.651603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:32.730215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f93159-bc61-4803-bdee-a52f09e2d954 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:32.810559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcfad6-c815-42bc-bb9d-3cb6b72bc6d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:32.811218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcfad6-c815-42bc-bb9d-3cb6b72bc6d7 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:32.811498 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcfad6-c815-42bc-bb9d-3cb6b72bc6d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:32.811769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcfad6-c815-42bc-bb9d-3cb6b72bc6d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:32.813495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1afcfad6-c815-42bc-bb9d-3cb6b72bc6d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:32.908194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:32.941724 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcfad6-c815-42bc-bb9d-3cb6b72bc6d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:32.943438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcfad6-c815-42bc-bb9d-3cb6b72bc6d7 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:32.964119 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 775/1490] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:43:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:43:35.271948 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244e92e-864d-4f9c-a55a-e0003a4f2150 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:35.272900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244e92e-864d-4f9c-a55a-e0003a4f2150 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:35.272900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244e92e-864d-4f9c-a55a-e0003a4f2150 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:35.274521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 716/1491] 10.222.0.70 () {54 vars in 926 bytes} [Tue Jul 19 20:43:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:43:35.280573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9f7b-f7db-4162-9e5c-1fdfcb746a7f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:35.281117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9f7b-f7db-4162-9e5c-1fdfcb746a7f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:35.281117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9f7b-f7db-4162-9e5c-1fdfcb746a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:35.378900 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-671f9f7b-f7db-4162-9e5c-1fdfcb746a7f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:35.380152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-671f9f7b-f7db-4162-9e5c-1fdfcb746a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:35.490430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 776/1492] 10.222.0.70 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:43:35.531146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244e92e-864d-4f9c-a55a-e0003a4f2150 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:35.532728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1244e92e-864d-4f9c-a55a-e0003a4f2150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:35.641389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244e92e-864d-4f9c-a55a-e0003a4f2150 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:35.755409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76036739-29c5-40d2-b7a7-cf288a978515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:35.756075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76036739-29c5-40d2-b7a7-cf288a978515 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:35.756354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76036739-29c5-40d2-b7a7-cf288a978515 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:35.756659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76036739-29c5-40d2-b7a7-cf288a978515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:35.758409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-76036739-29c5-40d2-b7a7-cf288a978515 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:35.768819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:35.885548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76036739-29c5-40d2-b7a7-cf288a978515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:35.887014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76036739-29c5-40d2-b7a7-cf288a978515 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:35.906573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 717/1493] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:43:45.487693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=82 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:45.904274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=50 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:05.634243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ed882-d9ba-46ef-b474-079d9de87a73 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:05.634822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ed882-d9ba-46ef-b474-079d9de87a73 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:05.634822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ed882-d9ba-46ef-b474-079d9de87a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:05.715220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-722ed882-d9ba-46ef-b474-079d9de87a73 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:05.716444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-722ed882-d9ba-46ef-b474-079d9de87a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:05.818412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 777/1494] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:45:05.826748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348933c-63ff-43c7-b561-4aa29fe8fc8f None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:05.827255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348933c-63ff-43c7-b561-4aa29fe8fc8f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:05.827358 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348933c-63ff-43c7-b561-4aa29fe8fc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:05.902613 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-b348933c-63ff-43c7-b561-4aa29fe8fc8f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:05.903752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b348933c-63ff-43c7-b561-4aa29fe8fc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:06.003666 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 718/1495] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:06.009859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ed882-d9ba-46ef-b474-079d9de87a73 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:06.011438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-722ed882-d9ba-46ef-b474-079d9de87a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:06.022897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ed882-d9ba-46ef-b474-079d9de87a73 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:06.084054 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:06.119590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cd8d3-14cc-43d6-a448-0ba1cd81f97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:06.120259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cd8d3-14cc-43d6-a448-0ba1cd81f97c admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:06.120542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cd8d3-14cc-43d6-a448-0ba1cd81f97c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:06.120818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cd8d3-14cc-43d6-a448-0ba1cd81f97c admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:06.121595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4cd8d3-14cc-43d6-a448-0ba1cd81f97c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:45:06.121905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4cd8d3-14cc-43d6-a448-0ba1cd81f97c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:06.122928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: WARNING py.warnings [None req-de4cd8d3-14cc-43d6-a448-0ba1cd81f97c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:45:06.122928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: warnings.warn(msg) Jul 19 20:45:06.122928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: Jul 19 20:45:06.123819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4cd8d3-14cc-43d6-a448-0ba1cd81f97c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:06.135482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 778/1496] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:45:06.144337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348933c-63ff-43c7-b561-4aa29fe8fc8f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:06.153829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348933c-63ff-43c7-b561-4aa29fe8fc8f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:06.216144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:06.250287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13629e6-38ca-4b7e-91df-026c0eb18b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:06.250945 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13629e6-38ca-4b7e-91df-026c0eb18b33 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:06.251227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13629e6-38ca-4b7e-91df-026c0eb18b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:06.251503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13629e6-38ca-4b7e-91df-026c0eb18b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:06.252204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13629e6-38ca-4b7e-91df-026c0eb18b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:06.253984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: WARNING py.warnings [None req-c13629e6-38ca-4b7e-91df-026c0eb18b33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:45:06.253984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: warnings.warn(msg) Jul 19 20:45:06.253984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: Jul 19 20:45:06.255174 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13629e6-38ca-4b7e-91df-026c0eb18b33 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:06.310396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 719/1497] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:45:06 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:45:06.314682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cd8d3-14cc-43d6-a448-0ba1cd81f97c admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:06.316256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-de4cd8d3-14cc-43d6-a448-0ba1cd81f97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:06.351310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cd8d3-14cc-43d6-a448-0ba1cd81f97c admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:06.463611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0826b-7cf7-44ef-b644-9e1cd5e6fc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:06.464204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0826b-7cf7-44ef-b644-9e1cd5e6fc72 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:06.464492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0826b-7cf7-44ef-b644-9e1cd5e6fc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:06.464767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0826b-7cf7-44ef-b644-9e1cd5e6fc72 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:06.465512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0826b-7cf7-44ef-b644-9e1cd5e6fc72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:06.466555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: WARNING py.warnings [None req-81d0826b-7cf7-44ef-b644-9e1cd5e6fc72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:45:06.466555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: warnings.warn(msg) Jul 19 20:45:06.466555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: Jul 19 20:45:06.467747 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0826b-7cf7-44ef-b644-9e1cd5e6fc72 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:06.516456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 779/1498] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:45:06 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:45:06.524422 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13629e6-38ca-4b7e-91df-026c0eb18b33 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:06.526003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c13629e6-38ca-4b7e-91df-026c0eb18b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:06.593601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13629e6-38ca-4b7e-91df-026c0eb18b33 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:06.682395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:06.694712 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fffa91-bc1d-41b8-9db5-f1b7bcb5a0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:06.695383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fffa91-bc1d-41b8-9db5-f1b7bcb5a0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:06.695659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fffa91-bc1d-41b8-9db5-f1b7bcb5a0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:06.695930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fffa91-bc1d-41b8-9db5-f1b7bcb5a0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:06.696656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fffa91-bc1d-41b8-9db5-f1b7bcb5a0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:06.697709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: WARNING py.warnings [None req-d1fffa91-bc1d-41b8-9db5-f1b7bcb5a0b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:45:06.697709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: warnings.warn(msg) Jul 19 20:45:06.697709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: Jul 19 20:45:06.698585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fffa91-bc1d-41b8-9db5-f1b7bcb5a0b2 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:06.712023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 720/1499] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:45:06.716466 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0826b-7cf7-44ef-b644-9e1cd5e6fc72 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:06.718238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-81d0826b-7cf7-44ef-b644-9e1cd5e6fc72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:06.729932 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0826b-7cf7-44ef-b644-9e1cd5e6fc72 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:06.762428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:06.848229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4304f27-4cbd-4e3e-a877-94c2256a9ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:06.848911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4304f27-4cbd-4e3e-a877-94c2256a9ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:06.849194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4304f27-4cbd-4e3e-a877-94c2256a9ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:06.849502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4304f27-4cbd-4e3e-a877-94c2256a9ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:06.850237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4304f27-4cbd-4e3e-a877-94c2256a9ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:06.851446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: WARNING py.warnings [None req-c4304f27-4cbd-4e3e-a877-94c2256a9ed6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:45:06.851446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: warnings.warn(msg) Jul 19 20:45:06.851446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: Jul 19 20:45:06.852161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4304f27-4cbd-4e3e-a877-94c2256a9ed6 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:06.865738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 780/1500] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:45:06.870031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fffa91-bc1d-41b8-9db5-f1b7bcb5a0b2 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:06.871630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d1fffa91-bc1d-41b8-9db5-f1b7bcb5a0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:06.906486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fffa91-bc1d-41b8-9db5-f1b7bcb5a0b2 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:07.011414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d3e8e-292b-4a63-bc9c-8bb7f1fa05f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:07.012222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d3e8e-292b-4a63-bc9c-8bb7f1fa05f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:07.012502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d3e8e-292b-4a63-bc9c-8bb7f1fa05f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:07.012787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d3e8e-292b-4a63-bc9c-8bb7f1fa05f9 admin admin] PATH_INFO: `/v3/projects/231060ca6c734146ab67844607145dd0/users/b43a2f3673e44ef3b6dad163fe71174e/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:07.035448 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d3e8e-292b-4a63-bc9c-8bb7f1fa05f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231060ca6c734146ab67844607145dd0, user_id=b43a2f3673e44ef3b6dad163fe71174e, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:07.036500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: WARNING py.warnings [None req-3e9d3e8e-292b-4a63-bc9c-8bb7f1fa05f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:45:07.036500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: warnings.warn(msg) Jul 19 20:45:07.036500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: Jul 19 20:45:07.038567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d3e8e-292b-4a63-bc9c-8bb7f1fa05f9 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:07.066857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 721/1501] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:45:06 2022] PUT /identity/v3/projects/231060ca6c734146ab67844607145dd0/users/b43a2f3673e44ef3b6dad163fe71174e/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:45:07.079243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff7ffe-ab88-4504-8ffc-475acfdb8a23 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:07.079537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff7ffe-ab88-4504-8ffc-475acfdb8a23 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:07.079823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff7ffe-ab88-4504-8ffc-475acfdb8a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:07.139640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-19ff7ffe-ab88-4504-8ffc-475acfdb8a23 None None] MFA Rules not processed for user `b43a2f3673e44ef3b6dad163fe71174e`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:07.140863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-19ff7ffe-ab88-4504-8ffc-475acfdb8a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:07.223721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 781/1502] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:45:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:07.237391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b7ade-accb-4c9a-a177-0cf31f550224 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:07.237686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b7ade-accb-4c9a-a177-0cf31f550224 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:07.237963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b7ade-accb-4c9a-a177-0cf31f550224 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:07.239398 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 722/1503] 10.222.0.70 () {54 vars in 964 bytes} [Tue Jul 19 20:45:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:07.244520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182d015-333d-475c-95f3-bab6d46ab4ee None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:07.244810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182d015-333d-475c-95f3-bab6d46ab4ee None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:07.245092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182d015-333d-475c-95f3-bab6d46ab4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:07.276128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:07.324355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-c182d015-333d-475c-95f3-bab6d46ab4ee None None] MFA Rules not processed for user `a75951e9512c420992a4fd6c1a6b1176`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:07.325642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c182d015-333d-475c-95f3-bab6d46ab4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:07.416059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 782/1504] 10.222.0.70 () {58 vars in 1061 bytes} [Tue Jul 19 20:45:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:07.424619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b7ade-accb-4c9a-a177-0cf31f550224 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:07.426246 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5f3b7ade-accb-4c9a-a177-0cf31f550224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:07.481412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:07.537692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b7ade-accb-4c9a-a177-0cf31f550224 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:07.611712 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53878-aca1-4047-8fec-5b3d763e92c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:07.612377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53878-aca1-4047-8fec-5b3d763e92c1 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:07.612655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53878-aca1-4047-8fec-5b3d763e92c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:07.612930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d53878-aca1-4047-8fec-5b3d763e92c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:07.614637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-44d53878-aca1-4047-8fec-5b3d763e92c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:07.716131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d53878-aca1-4047-8fec-5b3d763e92c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:07.717607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d53878-aca1-4047-8fec-5b3d763e92c1 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:07.721911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:07.737114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 723/1505] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:45:07.780052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9a0ee-c447-4877-8ca8-328c667faf86 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:07.780052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9a0ee-c447-4877-8ca8-328c667faf86 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:07.780493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9a0ee-c447-4877-8ca8-328c667faf86 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:07.781523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 783/1506] 10.222.0.70 () {52 vars in 829 bytes} [Tue Jul 19 20:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:07.785763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53878-aca1-4047-8fec-5b3d763e92c1 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:07.787009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-44d53878-aca1-4047-8fec-5b3d763e92c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:07.885051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d53878-aca1-4047-8fec-5b3d763e92c1 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:07.982910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:07.983813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:07.984092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:07.984367 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] PATH_INFO: `/v3/users/b43a2f3673e44ef3b6dad163fe71174e` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:07.997684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b43a2f3673e44ef3b6dad163fe71174e)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:07.998723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: WARNING py.warnings [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:45:07.998723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: warnings.warn(msg) Jul 19 20:45:07.998723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: Jul 19 20:45:07.999938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:08.011315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:08.072037 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b43a2f3673e44ef3b6dad163fe71174e'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:45:08.082513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b43a2f3673e44ef3b6dad163fe71174e'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:45:08.096271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b43a2f3673e44ef3b6dad163fe71174e'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:45:08.115009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b43a2f3673e44ef3b6dad163fe71174e'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:45:08.115490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b43a2f3673e44ef3b6dad163fe71174e'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:45:08.121430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 724/1507] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:45:07 2022] DELETE /identity/v3/users/b43a2f3673e44ef3b6dad163fe71174e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:45:08.125922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73772216-47df-4f14-a930-1356b3aafc4c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:08.126179 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73772216-47df-4f14-a930-1356b3aafc4c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:08.126452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73772216-47df-4f14-a930-1356b3aafc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:08.183412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-73772216-47df-4f14-a930-1356b3aafc4c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:08.184521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-73772216-47df-4f14-a930-1356b3aafc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:08.261636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:08.275206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 784/1508] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:08.284963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:08.286248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:08.396515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df386b-3799-40e5-a9c4-bb2a9cd4846a admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:08.460184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:08.482730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143dec34-ff95-46f9-9f1f-556bde265d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:08.483398 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dec34-ff95-46f9-9f1f-556bde265d08 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:08.483709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dec34-ff95-46f9-9f1f-556bde265d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:08.483987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dec34-ff95-46f9-9f1f-556bde265d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:08.485703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-143dec34-ff95-46f9-9f1f-556bde265d08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:08.594174 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143dec34-ff95-46f9-9f1f-556bde265d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:08.595647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143dec34-ff95-46f9-9f1f-556bde265d08 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:08.614815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 725/1509] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:45:08.954012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73772216-47df-4f14-a930-1356b3aafc4c None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:08.955324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-73772216-47df-4f14-a930-1356b3aafc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:08.967527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73772216-47df-4f14-a930-1356b3aafc4c None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:09.069441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:09.070678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:09.070960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:09.071265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] PATH_INFO: `/v3/projects/231060ca6c734146ab67844607145dd0` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:09.081999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] RBAC: Authorizing `identity:delete_project(project_id=231060ca6c734146ab67844607145dd0)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:09.083053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: WARNING py.warnings [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:45:09.083053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: warnings.warn(msg) Jul 19 20:45:09.083053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: Jul 19 20:45:09.084269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:09.145726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231060ca6c734146ab67844607145dd0', 'request_id': 'req-72cd5901-2b0d-4655-87dc-573a6f9cf1da'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:45:09.155779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231060ca6c734146ab67844607145dd0', 'request_id': 'req-72cd5901-2b0d-4655-87dc-573a6f9cf1da'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:45:09.162897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231060ca6c734146ab67844607145dd0', 'request_id': 'req-72cd5901-2b0d-4655-87dc-573a6f9cf1da'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:45:09.193534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] The token cache is being invalidate because project 231060ca6c734146ab67844607145dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:45:09.193827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:45:09.194268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-72cd5901-2b0d-4655-87dc-573a6f9cf1da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:45:09.195739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 785/1510] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:45:08 2022] DELETE /identity/v3/projects/231060ca6c734146ab67844607145dd0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:45:18.612474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=62 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:19.191521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:54.412040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20bfe6-0c97-4308-b125-c68140887619 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:54.412734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20bfe6-0c97-4308-b125-c68140887619 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:54.412734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20bfe6-0c97-4308-b125-c68140887619 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:54.414229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 726/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:46:54.419115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c054a15-8ec2-4a1f-af12-e5226980e0b6 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:54.419629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c054a15-8ec2-4a1f-af12-e5226980e0b6 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:54.419736 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c054a15-8ec2-4a1f-af12-e5226980e0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:54.504166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-9c054a15-8ec2-4a1f-af12-e5226980e0b6 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:54.505457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9c054a15-8ec2-4a1f-af12-e5226980e0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:54.616989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 786/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:54.779774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20bfe6-0c97-4308-b125-c68140887619 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:54.782723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6f20bfe6-0c97-4308-b125-c68140887619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:54.911985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20bfe6-0c97-4308-b125-c68140887619 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:54.917626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb0b467-e6c9-43e8-9d6f-f890b3c38e66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:54.918281 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0b467-e6c9-43e8-9d6f-f890b3c38e66 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:54.918567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0b467-e6c9-43e8-9d6f-f890b3c38e66 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:54.918840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0b467-e6c9-43e8-9d6f-f890b3c38e66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:54.928551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb0b467-e6c9-43e8-9d6f-f890b3c38e66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:54.930179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb0b467-e6c9-43e8-9d6f-f890b3c38e66 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:54.941523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 727/1513] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:46:55.805439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4a21c-c3ec-4e68-8c2e-e5ef7b521cc1 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:55.806006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4a21c-c3ec-4e68-8c2e-e5ef7b521cc1 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:55.806006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4a21c-c3ec-4e68-8c2e-e5ef7b521cc1 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:55.807417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 787/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:46:55.812945 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45d2d4-fdaa-4bd5-951b-21d0e8d9dc34 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:55.813646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45d2d4-fdaa-4bd5-951b-21d0e8d9dc34 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:55.813646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45d2d4-fdaa-4bd5-951b-21d0e8d9dc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:55.834519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-1b45d2d4-fdaa-4bd5-951b-21d0e8d9dc34 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:55.835754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1b45d2d4-fdaa-4bd5-951b-21d0e8d9dc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:55.844177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 728/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:56.018420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4a21c-c3ec-4e68-8c2e-e5ef7b521cc1 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:56.034430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4a21c-c3ec-4e68-8c2e-e5ef7b521cc1 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:56.040308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523a7c2-0a5a-4c5c-a0e8-aa8e332b8c96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:56.040897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523a7c2-0a5a-4c5c-a0e8-aa8e332b8c96 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:56.041232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523a7c2-0a5a-4c5c-a0e8-aa8e332b8c96 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:56.041594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523a7c2-0a5a-4c5c-a0e8-aa8e332b8c96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:56.051805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523a7c2-0a5a-4c5c-a0e8-aa8e332b8c96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:56.053460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523a7c2-0a5a-4c5c-a0e8-aa8e332b8c96 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:56.064940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 788/1516] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:56.504344 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0da1cc-4473-4899-b583-3906a9f2eec1 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:56.504644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0da1cc-4473-4899-b583-3906a9f2eec1 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:56.504922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0da1cc-4473-4899-b583-3906a9f2eec1 None None] PATH_INFO: `/v3` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:56.506283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 729/1517] 10.222.0.84 () {54 vars in 925 bytes} [Tue Jul 19 20:46:56 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:46:56.512629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec5d67-6d01-4383-98ad-8dad30f8c6e4 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:56.512914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec5d67-6d01-4383-98ad-8dad30f8c6e4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:56.513194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec5d67-6d01-4383-98ad-8dad30f8c6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:56.573619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-beec5d67-6d01-4383-98ad-8dad30f8c6e4 None None] MFA Rules not processed for user `f27871b39b9b4edaa512bd5b7467174a`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:56.574871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-beec5d67-6d01-4383-98ad-8dad30f8c6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:56.595586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:56.666497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 789/1518] 10.222.0.84 () {58 vars in 1015 bytes} [Tue Jul 19 20:46:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:00.585219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7a3c-83ff-4bd6-aeb3-8a69d6957432 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:00.585820 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7a3c-83ff-4bd6-aeb3-8a69d6957432 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:00.585820 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7a3c-83ff-4bd6-aeb3-8a69d6957432 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:00.587075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 730/1519] 10.222.0.70 () {54 vars in 961 bytes} [Tue Jul 19 20:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:00.592648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23393679-fc5d-47ab-8327-923f46f5f346 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:00.593121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23393679-fc5d-47ab-8327-923f46f5f346 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:00.593240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23393679-fc5d-47ab-8327-923f46f5f346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:00.614744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-23393679-fc5d-47ab-8327-923f46f5f346 None None] MFA Rules not processed for user `f27871b39b9b4edaa512bd5b7467174a`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:00.615979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-23393679-fc5d-47ab-8327-923f46f5f346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:00.624838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 790/1520] 10.222.0.70 () {58 vars in 1058 bytes} [Tue Jul 19 20:47:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:00.632399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db7a3c-83ff-4bd6-aeb3-8a69d6957432 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:00.646061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db7a3c-83ff-4bd6-aeb3-8a69d6957432 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:00.651230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43872cdf-e514-45ad-b03c-cf03d0d3f1d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:00.651879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43872cdf-e514-45ad-b03c-cf03d0d3f1d4 service placement] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:00.652166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43872cdf-e514-45ad-b03c-cf03d0d3f1d4 service placement] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:00.652443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43872cdf-e514-45ad-b03c-cf03d0d3f1d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:00.662044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43872cdf-e514-45ad-b03c-cf03d0d3f1d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:00.663642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43872cdf-e514-45ad-b03c-cf03d0d3f1d4 service placement] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:00.674165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 731/1521] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:07.320976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b6835-2b5d-4194-9bab-5b5970319510 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:07.321563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b6835-2b5d-4194-9bab-5b5970319510 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:07.321563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b6835-2b5d-4194-9bab-5b5970319510 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:07.322984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 791/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:07.328061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1e067-6ad5-4391-b560-6047c664b1c8 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:07.328498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1e067-6ad5-4391-b560-6047c664b1c8 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:07.328607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1e067-6ad5-4391-b560-6047c664b1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:07.347518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-50c1e067-6ad5-4391-b560-6047c664b1c8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:07.348726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-50c1e067-6ad5-4391-b560-6047c664b1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:07.356697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 732/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:07.516219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b6835-2b5d-4194-9bab-5b5970319510 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:07.531563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b6835-2b5d-4194-9bab-5b5970319510 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:07.536959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cb56e-f317-4798-bcdc-ba059a456a66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:07.537680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cb56e-f317-4798-bcdc-ba059a456a66 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:07.537960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cb56e-f317-4798-bcdc-ba059a456a66 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:07.538231 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cb56e-f317-4798-bcdc-ba059a456a66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:07.546491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cb56e-f317-4798-bcdc-ba059a456a66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:07.548061 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cb56e-f317-4798-bcdc-ba059a456a66 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:07.557667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 792/1524] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:08.399452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778a0e2-7b90-4b9c-b5b5-d49513656316 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:08.400101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778a0e2-7b90-4b9c-b5b5-d49513656316 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:08.400101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778a0e2-7b90-4b9c-b5b5-d49513656316 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:08.401441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 733/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:08.406525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24972f20-5746-44f5-a652-a88b49a9afcf None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:08.407554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24972f20-5746-44f5-a652-a88b49a9afcf None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:08.407749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24972f20-5746-44f5-a652-a88b49a9afcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:08.545457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-24972f20-5746-44f5-a652-a88b49a9afcf None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:08.546708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-24972f20-5746-44f5-a652-a88b49a9afcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:08.555007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 793/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:08.707433 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778a0e2-7b90-4b9c-b5b5-d49513656316 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:08.722645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778a0e2-7b90-4b9c-b5b5-d49513656316 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:08.728114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1aab6-c89d-4a1b-b7e0-6e03a3a4b7ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:08.728763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1aab6-c89d-4a1b-b7e0-6e03a3a4b7ff service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:08.729049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1aab6-c89d-4a1b-b7e0-6e03a3a4b7ff service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:08.729330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1aab6-c89d-4a1b-b7e0-6e03a3a4b7ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:08.738337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1aab6-c89d-4a1b-b7e0-6e03a3a4b7ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:08.739956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1aab6-c89d-4a1b-b7e0-6e03a3a4b7ff service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:08.749543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 734/1527] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:12.751301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37da454-b036-439b-a8e6-4533da9d4787 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:12.751883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37da454-b036-439b-a8e6-4533da9d4787 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:12.751883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37da454-b036-439b-a8e6-4533da9d4787 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:12.753132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 794/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:12.758704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82eca-daa7-47b1-b408-9393bcb7a1de None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:12.759230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82eca-daa7-47b1-b408-9393bcb7a1de None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:12.759363 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82eca-daa7-47b1-b408-9393bcb7a1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:12.777925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-78f82eca-daa7-47b1-b408-9393bcb7a1de None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:12.779135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-78f82eca-daa7-47b1-b408-9393bcb7a1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:12.789739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 735/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:12.875664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37da454-b036-439b-a8e6-4533da9d4787 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:12.877418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d37da454-b036-439b-a8e6-4533da9d4787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:12.964846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37da454-b036-439b-a8e6-4533da9d4787 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:12.968457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908ae9b-77ce-4bef-bccf-6c5d5b8568fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:12.969106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908ae9b-77ce-4bef-bccf-6c5d5b8568fc service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:12.969432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908ae9b-77ce-4bef-bccf-6c5d5b8568fc service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:12.969718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908ae9b-77ce-4bef-bccf-6c5d5b8568fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:12.979592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908ae9b-77ce-4bef-bccf-6c5d5b8568fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:12.981019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908ae9b-77ce-4bef-bccf-6c5d5b8568fc service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:12.991465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 795/1530] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:15.033946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16521dd2-5b49-4572-b380-485ac0b49253 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:15.034467 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16521dd2-5b49-4572-b380-485ac0b49253 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:15.034467 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16521dd2-5b49-4572-b380-485ac0b49253 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:15.035834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 736/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:15.041471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe86650-e2bf-4c6f-9958-238e92e28f5d None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:15.041939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe86650-e2bf-4c6f-9958-238e92e28f5d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:15.042054 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe86650-e2bf-4c6f-9958-238e92e28f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:15.060900 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-efe86650-e2bf-4c6f-9958-238e92e28f5d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:15.062148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-efe86650-e2bf-4c6f-9958-238e92e28f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:15.071751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 796/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:15.110683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16521dd2-5b49-4572-b380-485ac0b49253 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:15.112931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-16521dd2-5b49-4572-b380-485ac0b49253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:15.126682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16521dd2-5b49-4572-b380-485ac0b49253 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:15.130795 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd5fac-cb82-4758-b4a4-74ab82a390c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:15.131462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5fac-cb82-4758-b4a4-74ab82a390c8 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:15.131745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5fac-cb82-4758-b4a4-74ab82a390c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:15.132020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5fac-cb82-4758-b4a4-74ab82a390c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:15.142476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd5fac-cb82-4758-b4a4-74ab82a390c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:15.143898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd5fac-cb82-4758-b4a4-74ab82a390c8 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:15.154140 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 737/1533] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:18.632729 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82d553-3bed-4904-afea-302015c5c1f3 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:18.633309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82d553-3bed-4904-afea-302015c5c1f3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:18.633309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82d553-3bed-4904-afea-302015c5c1f3 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:18.634686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 797/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:18.640644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f13d70-8533-4114-82e8-b3db5e6c7005 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:18.641806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f13d70-8533-4114-82e8-b3db5e6c7005 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:18.641914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f13d70-8533-4114-82e8-b3db5e6c7005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:18.648444 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=81 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:18.670480 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c0f13d70-8533-4114-82e8-b3db5e6c7005 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:18.671875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c0f13d70-8533-4114-82e8-b3db5e6c7005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:18.680183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 738/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:19.029387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82d553-3bed-4904-afea-302015c5c1f3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:19.031245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fc82d553-3bed-4904-afea-302015c5c1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:19.106687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:19.123012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82d553-3bed-4904-afea-302015c5c1f3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:19.127766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae1259-43af-429d-817d-d0249dbf3e7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:19.128433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae1259-43af-429d-817d-d0249dbf3e7c service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:19.128719 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae1259-43af-429d-817d-d0249dbf3e7c service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:19.128993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae1259-43af-429d-817d-d0249dbf3e7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:19.139220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae1259-43af-429d-817d-d0249dbf3e7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:19.140657 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae1259-43af-429d-817d-d0249dbf3e7c service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:19.151706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 798/1536] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:21.248623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626731c8-1ab7-42ac-804c-b93043d01610 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:21.249227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626731c8-1ab7-42ac-804c-b93043d01610 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:21.249227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626731c8-1ab7-42ac-804c-b93043d01610 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:21.251601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 739/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:21.256646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3bf1d-3886-4414-8610-ac12669e3098 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:21.257103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3bf1d-3886-4414-8610-ac12669e3098 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:21.257218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3bf1d-3886-4414-8610-ac12669e3098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:21.274672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-91a3bf1d-3886-4414-8610-ac12669e3098 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:21.275885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-91a3bf1d-3886-4414-8610-ac12669e3098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:21.283933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 799/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:21.592999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626731c8-1ab7-42ac-804c-b93043d01610 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:21.595205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-626731c8-1ab7-42ac-804c-b93043d01610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:21.608227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626731c8-1ab7-42ac-804c-b93043d01610 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:21.612463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38949a-91c4-47ba-888e-6e1280ba6206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:21.613106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38949a-91c4-47ba-888e-6e1280ba6206 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:21.613422 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38949a-91c4-47ba-888e-6e1280ba6206 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:21.613707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38949a-91c4-47ba-888e-6e1280ba6206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:21.623268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38949a-91c4-47ba-888e-6e1280ba6206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:21.624708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38949a-91c4-47ba-888e-6e1280ba6206 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:21.634546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 740/1539] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:31.269808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=11 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:31.632416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=8 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:52.616535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17a925-aac9-4252-a7cd-92f34cd3e91b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.617195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17a925-aac9-4252-a7cd-92f34cd3e91b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.617195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17a925-aac9-4252-a7cd-92f34cd3e91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.646674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-af17a925-aac9-4252-a7cd-92f34cd3e91b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.647926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-af17a925-aac9-4252-a7cd-92f34cd3e91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.648844 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fa7c7-c97c-43b9-98b5-334c4a98e88b None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.649787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fa7c7-c97c-43b9-98b5-334c4a98e88b None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.650447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fa7c7-c97c-43b9-98b5-334c4a98e88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.661402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 800/1540] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:52.674273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c21e27-9373-4d82-9951-4baeb9738adf None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.674620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c21e27-9373-4d82-9951-4baeb9738adf None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.674937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c21e27-9373-4d82-9951-4baeb9738adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.678755 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c81fa7c7-c97c-43b9-98b5-334c4a98e88b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.680252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c81fa7c7-c97c-43b9-98b5-334c4a98e88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.695337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 741/1541] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:52.704184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72a22-2dce-45d8-83d3-74fa0a9484ad None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.705432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72a22-2dce-45d8-83d3-74fa0a9484ad None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.706434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72a22-2dce-45d8-83d3-74fa0a9484ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.716012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-b6c21e27-9373-4d82-9951-4baeb9738adf None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.716012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b6c21e27-9373-4d82-9951-4baeb9738adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.730725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 801/1542] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:48:52.732038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39305636-bbb9-425d-b5a4-87dc4e8031a8 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.732329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39305636-bbb9-425d-b5a4-87dc4e8031a8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.732611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39305636-bbb9-425d-b5a4-87dc4e8031a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.738492 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-aff72a22-2dce-45d8-83d3-74fa0a9484ad None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.739723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-aff72a22-2dce-45d8-83d3-74fa0a9484ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.758494 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 742/1543] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:48:52.763164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cf590-a69f-400e-b72f-3798063eb36c None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.763457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cf590-a69f-400e-b72f-3798063eb36c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.763736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cf590-a69f-400e-b72f-3798063eb36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.771470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-39305636-bbb9-425d-b5a4-87dc4e8031a8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.775614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-39305636-bbb9-425d-b5a4-87dc4e8031a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.791476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 802/1544] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:52.792430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39305636-bbb9-425d-b5a4-87dc4e8031a8 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:52.794202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-978cf590-a69f-400e-b72f-3798063eb36c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.795435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-978cf590-a69f-400e-b72f-3798063eb36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.807203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 743/1545] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:52.807990 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39305636-bbb9-425d-b5a4-87dc4e8031a8 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:52.808531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca984d67-0036-4c8f-908f-fa5d9c518e07 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.808918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca984d67-0036-4c8f-908f-fa5d9c518e07 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.809211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca984d67-0036-4c8f-908f-fa5d9c518e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.819349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc83cc-14f7-4943-b5c5-294b8923a5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:52.820395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc83cc-14f7-4943-b5c5-294b8923a5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.820848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc83cc-14f7-4943-b5c5-294b8923a5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.821257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc83cc-14f7-4943-b5c5-294b8923a5e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.822503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc83cc-14f7-4943-b5c5-294b8923a5e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:48:52.822991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc83cc-14f7-4943-b5c5-294b8923a5e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:52.824916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc83cc-14f7-4943-b5c5-294b8923a5e9 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:52.833033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ca984d67-0036-4c8f-908f-fa5d9c518e07 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.834317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ca984d67-0036-4c8f-908f-fa5d9c518e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.837811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 803/1546] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:48:52.838917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc83cc-14f7-4943-b5c5-294b8923a5e9 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:52.848620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 744/1547] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:52.849961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc673712-f280-426c-9541-f13f1e11dbc1 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.850371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc673712-f280-426c-9541-f13f1e11dbc1 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.850650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc673712-f280-426c-9541-f13f1e11dbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.854061 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc83cc-14f7-4943-b5c5-294b8923a5e9 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:52.860297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b1729-67da-46e9-bcfe-4a66213d2c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:52.861267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b1729-67da-46e9-bcfe-4a66213d2c40 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.861779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b1729-67da-46e9-bcfe-4a66213d2c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.862062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b1729-67da-46e9-bcfe-4a66213d2c40 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.862807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b1729-67da-46e9-bcfe-4a66213d2c40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:48:52.863118 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b1729-67da-46e9-bcfe-4a66213d2c40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:52.864725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b1729-67da-46e9-bcfe-4a66213d2c40 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:52.871646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-cc673712-f280-426c-9541-f13f1e11dbc1 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.873016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-cc673712-f280-426c-9541-f13f1e11dbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.878165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 804/1548] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:48:52.879272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a466752-06dd-41a6-8218-4b6efa4414f8 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.879562 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a466752-06dd-41a6-8218-4b6efa4414f8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.879843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a466752-06dd-41a6-8218-4b6efa4414f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.893884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 745/1549] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:52.893884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5888d-417c-4bfe-a81f-bb2865faaf40 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.893884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5888d-417c-4bfe-a81f-bb2865faaf40 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.893884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5888d-417c-4bfe-a81f-bb2865faaf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.900564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-5a466752-06dd-41a6-8218-4b6efa4414f8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.901761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5a466752-06dd-41a6-8218-4b6efa4414f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.910894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 805/1550] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:52.912004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f54e8-8ae4-4237-a002-b7e3c8e6575f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.912298 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f54e8-8ae4-4237-a002-b7e3c8e6575f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.912577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f54e8-8ae4-4237-a002-b7e3c8e6575f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.919009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c3e5888d-417c-4bfe-a81f-bb2865faaf40 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.920544 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c3e5888d-417c-4bfe-a81f-bb2865faaf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.929674 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 746/1551] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:52.930930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5888d-417c-4bfe-a81f-bb2865faaf40 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:52.936203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-772f54e8-8ae4-4237-a002-b7e3c8e6575f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.937460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-772f54e8-8ae4-4237-a002-b7e3c8e6575f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.942083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5888d-417c-4bfe-a81f-bb2865faaf40 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:52.948825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2969cb9-f1aa-4da0-9010-a70b6aa3d54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:52.949680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2969cb9-f1aa-4da0-9010-a70b6aa3d54c admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.950044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2969cb9-f1aa-4da0-9010-a70b6aa3d54c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.950465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2969cb9-f1aa-4da0-9010-a70b6aa3d54c admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.956718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 806/1552] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:52.956959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2969cb9-f1aa-4da0-9010-a70b6aa3d54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:52.957101 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e9ac2-a75a-47cd-8dba-9b6b226f1383 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:52.957101 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e9ac2-a75a-47cd-8dba-9b6b226f1383 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:52.957101 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e9ac2-a75a-47cd-8dba-9b6b226f1383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:52.957498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2969cb9-f1aa-4da0-9010-a70b6aa3d54c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:52.977392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-1c4e9ac2-a75a-47cd-8dba-9b6b226f1383 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:52.978734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1c4e9ac2-a75a-47cd-8dba-9b6b226f1383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:52.987493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 807/1553] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:52.988412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e9ac2-a75a-47cd-8dba-9b6b226f1383 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.000037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e9ac2-a75a-47cd-8dba-9b6b226f1383 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.005762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1af6-b19c-4e99-b8dd-603384ab279b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.006428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1af6-b19c-4e99-b8dd-603384ab279b admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.006706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1af6-b19c-4e99-b8dd-603384ab279b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.006975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1af6-b19c-4e99-b8dd-603384ab279b admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.010832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 747/1554] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.011033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff1af6-b19c-4e99-b8dd-603384ab279b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.011146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121a06d-5999-496b-a240-9711acde51fc None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.011249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: WARNING py.warnings [None req-85ff1af6-b19c-4e99-b8dd-603384ab279b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:48:53.011249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: warnings.warn(msg) Jul 19 20:48:53.011249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: Jul 19 20:48:53.011509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121a06d-5999-496b-a240-9711acde51fc None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.011509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121a06d-5999-496b-a240-9711acde51fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.011677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff1af6-b19c-4e99-b8dd-603384ab279b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.031028 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-8121a06d-5999-496b-a240-9711acde51fc None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:53.032259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8121a06d-5999-496b-a240-9711acde51fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:53.044134 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 808/1555] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:48:52 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.044134 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1af6-b19c-4e99-b8dd-603384ab279b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.099860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 748/1556] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:53.101153 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf7e4a-ae38-4bb9-a92e-e4bb30fef6ce None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.101477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf7e4a-ae38-4bb9-a92e-e4bb30fef6ce None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.101756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf7e4a-ae38-4bb9-a92e-e4bb30fef6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.117011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1af6-b19c-4e99-b8dd-603384ab279b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.121837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cfab2-3d51-431b-88d3-8a2d42a7e57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.122056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-3fdf7e4a-ae38-4bb9-a92e-e4bb30fef6ce None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:53.122517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cfab2-3d51-431b-88d3-8a2d42a7e57e admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.122798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cfab2-3d51-431b-88d3-8a2d42a7e57e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.123074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cfab2-3d51-431b-88d3-8a2d42a7e57e admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.123278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3fdf7e4a-ae38-4bb9-a92e-e4bb30fef6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:53.123820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cfab2-3d51-431b-88d3-8a2d42a7e57e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:48:53.124125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cfab2-3d51-431b-88d3-8a2d42a7e57e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.125752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cfab2-3d51-431b-88d3-8a2d42a7e57e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.140937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 809/1557] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:48:53.142493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808bb0e-0f72-469b-8eb5-d6b6870c6ae4 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.143120 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808bb0e-0f72-469b-8eb5-d6b6870c6ae4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.143408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808bb0e-0f72-469b-8eb5-d6b6870c6ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.150364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 749/1558] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:53.151555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf7e4a-ae38-4bb9-a92e-e4bb30fef6ce None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.161971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf7e4a-ae38-4bb9-a92e-e4bb30fef6ce None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.163800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f808bb0e-0f72-469b-8eb5-d6b6870c6ae4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:53.165008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f808bb0e-0f72-469b-8eb5-d6b6870c6ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:53.169560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40995e2-064e-4089-a797-3498575d5ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.170258 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40995e2-064e-4089-a797-3498575d5ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.170538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40995e2-064e-4089-a797-3498575d5ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.170814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40995e2-064e-4089-a797-3498575d5ae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.171554 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40995e2-064e-4089-a797-3498575d5ae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:48:53.171890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40995e2-064e-4089-a797-3498575d5ae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.172936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: WARNING py.warnings [None req-e40995e2-064e-4089-a797-3498575d5ae1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:48:53.172936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: warnings.warn(msg) Jul 19 20:48:53.172936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: Jul 19 20:48:53.173228 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 810/1559] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:53.174468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808bb0e-0f72-469b-8eb5-d6b6870c6ae4 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.174813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40995e2-064e-4089-a797-3498575d5ae1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.186540 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808bb0e-0f72-469b-8eb5-d6b6870c6ae4 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.189021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 750/1560] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:48:53.190571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40995e2-064e-4089-a797-3498575d5ae1 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.194499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17c431-9eba-4499-9ce2-252e5ab70935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.195619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c431-9eba-4499-9ce2-252e5ab70935 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.196094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c431-9eba-4499-9ce2-252e5ab70935 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.196566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c431-9eba-4499-9ce2-252e5ab70935 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.197847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17c431-9eba-4499-9ce2-252e5ab70935 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:48:53.198330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17c431-9eba-4499-9ce2-252e5ab70935 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.200876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17c431-9eba-4499-9ce2-252e5ab70935 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.201321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40995e2-064e-4089-a797-3498575d5ae1 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.206216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec0070-c0e7-4228-84cb-e87c199c5b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.206884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec0070-c0e7-4228-84cb-e87c199c5b53 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.207164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec0070-c0e7-4228-84cb-e87c199c5b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.207575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec0070-c0e7-4228-84cb-e87c199c5b53 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.208275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec0070-c0e7-4228-84cb-e87c199c5b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.209285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: WARNING py.warnings [None req-d2ec0070-c0e7-4228-84cb-e87c199c5b53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:48:53.209285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: warnings.warn(msg) Jul 19 20:48:53.209285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: Jul 19 20:48:53.210507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec0070-c0e7-4228-84cb-e87c199c5b53 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.213580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 811/1561] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:48:53.214206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17c431-9eba-4499-9ce2-252e5ab70935 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.227713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17c431-9eba-4499-9ce2-252e5ab70935 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.233389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 751/1562] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.233964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec0070-c0e7-4228-84cb-e87c199c5b53 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.239245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1cbce-0e77-44b1-b439-d2d8b667c36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.239928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1cbce-0e77-44b1-b439-d2d8b667c36c admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.240208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1cbce-0e77-44b1-b439-d2d8b667c36c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.240492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1cbce-0e77-44b1-b439-d2d8b667c36c admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.241225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1cbce-0e77-44b1-b439-d2d8b667c36c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.243376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1cbce-0e77-44b1-b439-d2d8b667c36c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.244995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec0070-c0e7-4228-84cb-e87c199c5b53 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.254082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc7bfc-9fe5-4ec6-a375-b5f30ca5b4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.255241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7bfc-9fe5-4ec6-a375-b5f30ca5b4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.255519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7bfc-9fe5-4ec6-a375-b5f30ca5b4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.256065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7bfc-9fe5-4ec6-a375-b5f30ca5b4a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.256856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc7bfc-9fe5-4ec6-a375-b5f30ca5b4a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:48:53.257340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc7bfc-9fe5-4ec6-a375-b5f30ca5b4a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.259384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc7bfc-9fe5-4ec6-a375-b5f30ca5b4a1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.268710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 812/1563] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/users => generated 297 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.269640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1cbce-0e77-44b1-b439-d2d8b667c36c admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.273703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 752/1564] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:48:53.275600 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e6d9b-77ad-48a4-9a1f-84506e5c8d22 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.276007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e6d9b-77ad-48a4-9a1f-84506e5c8d22 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.276385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e6d9b-77ad-48a4-9a1f-84506e5c8d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.289225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1cbce-0e77-44b1-b439-d2d8b667c36c admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.294062 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-cb0e6d9b-77ad-48a4-9a1f-84506e5c8d22 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:53.294830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453671d-99ed-45ed-a368-c00a9477e2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.295282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-cb0e6d9b-77ad-48a4-9a1f-84506e5c8d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:53.295489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453671d-99ed-45ed-a368-c00a9477e2fc admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.295772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453671d-99ed-45ed-a368-c00a9477e2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.296046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453671d-99ed-45ed-a368-c00a9477e2fc admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.296733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453671d-99ed-45ed-a368-c00a9477e2fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.299187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453671d-99ed-45ed-a368-c00a9477e2fc admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.305347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 753/1565] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:53.306203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e6d9b-77ad-48a4-9a1f-84506e5c8d22 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.316496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e6d9b-77ad-48a4-9a1f-84506e5c8d22 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.323115 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad541e7-1f84-4cee-b054-3125f230584a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.323850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad541e7-1f84-4cee-b054-3125f230584a admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.324135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad541e7-1f84-4cee-b054-3125f230584a admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.324413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad541e7-1f84-4cee-b054-3125f230584a admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.325157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad541e7-1f84-4cee-b054-3125f230584a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:48:53.325516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad541e7-1f84-4cee-b054-3125f230584a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.327219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad541e7-1f84-4cee-b054-3125f230584a admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.329194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 813/1566] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.332784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453671d-99ed-45ed-a368-c00a9477e2fc admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.343874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:53.372011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 754/1567] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:48:53.372011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad541e7-1f84-4cee-b054-3125f230584a admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.383233 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad541e7-1f84-4cee-b054-3125f230584a admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.401382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:53.417718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453671d-99ed-45ed-a368-c00a9477e2fc admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.429878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f946fc-23d2-46db-a498-f7b9607b0591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.430947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f946fc-23d2-46db-a498-f7b9607b0591 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.430947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f946fc-23d2-46db-a498-f7b9607b0591 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.431158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f946fc-23d2-46db-a498-f7b9607b0591 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.431792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f946fc-23d2-46db-a498-f7b9607b0591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.433752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f946fc-23d2-46db-a498-f7b9607b0591 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.460492 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc51ed-04a1-4569-9238-a48a92c0fb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.461168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc51ed-04a1-4569-9238-a48a92c0fb12 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.461494 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc51ed-04a1-4569-9238-a48a92c0fb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.461772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc51ed-04a1-4569-9238-a48a92c0fb12 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.462471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc51ed-04a1-4569-9238-a48a92c0fb12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.464381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc51ed-04a1-4569-9238-a48a92c0fb12 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.477502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 814/1568] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.477502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f946fc-23d2-46db-a498-f7b9607b0591 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.499260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 755/1569] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.500400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc51ed-04a1-4569-9238-a48a92c0fb12 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.544244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f946fc-23d2-46db-a498-f7b9607b0591 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.579892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc51ed-04a1-4569-9238-a48a92c0fb12 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.586149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de97da-748f-4424-80fd-a5658d31e13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.586846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de97da-748f-4424-80fd-a5658d31e13d admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.587136 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de97da-748f-4424-80fd-a5658d31e13d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.587421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de97da-748f-4424-80fd-a5658d31e13d admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.588174 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de97da-748f-4424-80fd-a5658d31e13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.590474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de97da-748f-4424-80fd-a5658d31e13d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.603613 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 756/1570] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:53.604305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de97da-748f-4424-80fd-a5658d31e13d admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.614311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bfef4-7c13-444f-9771-40e426fd28b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.615511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bfef4-7c13-444f-9771-40e426fd28b8 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.615716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de97da-748f-4424-80fd-a5658d31e13d admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.615942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bfef4-7c13-444f-9771-40e426fd28b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.616370 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bfef4-7c13-444f-9771-40e426fd28b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.618228 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0bfef4-7c13-444f-9771-40e426fd28b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.620537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017d546-04cd-44f3-a457-e895ac6f2602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.620684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0bfef4-7c13-444f-9771-40e426fd28b8 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.621200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017d546-04cd-44f3-a457-e895ac6f2602 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.621516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017d546-04cd-44f3-a457-e895ac6f2602 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.621797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017d546-04cd-44f3-a457-e895ac6f2602 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.622501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017d546-04cd-44f3-a457-e895ac6f2602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.624404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017d546-04cd-44f3-a457-e895ac6f2602 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.632471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 815/1571] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:53.636270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bfef4-7c13-444f-9771-40e426fd28b8 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.646867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bfef4-7c13-444f-9771-40e426fd28b8 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.651509 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6d074-d6b8-4c08-a849-bc627eeff927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.652177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6d074-d6b8-4c08-a849-bc627eeff927 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.652462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6d074-d6b8-4c08-a849-bc627eeff927 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.652736 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6d074-d6b8-4c08-a849-bc627eeff927 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.653488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6d074-d6b8-4c08-a849-bc627eeff927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:48:53.653795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6d074-d6b8-4c08-a849-bc627eeff927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.655382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6d074-d6b8-4c08-a849-bc627eeff927 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.658786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 757/1572] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.659619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017d546-04cd-44f3-a457-e895ac6f2602 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.672447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 816/1573] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:48:53.678599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6d074-d6b8-4c08-a849-bc627eeff927 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.693806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6d074-d6b8-4c08-a849-bc627eeff927 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.721038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:53.748474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017d546-04cd-44f3-a457-e895ac6f2602 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.754196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceceeade-8be4-4a9b-94d0-edfd764cfee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.754950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceceeade-8be4-4a9b-94d0-edfd764cfee2 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.755287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceceeade-8be4-4a9b-94d0-edfd764cfee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.756131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceceeade-8be4-4a9b-94d0-edfd764cfee2 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.756572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceceeade-8be4-4a9b-94d0-edfd764cfee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.758732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceceeade-8be4-4a9b-94d0-edfd764cfee2 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.782582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb76fb-081d-48c8-8f38-23ebdd55e243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.784210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb76fb-081d-48c8-8f38-23ebdd55e243 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.784659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb76fb-081d-48c8-8f38-23ebdd55e243 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.785108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb76fb-081d-48c8-8f38-23ebdd55e243 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.785871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 758/1574] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.786296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb76fb-081d-48c8-8f38-23ebdd55e243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.790138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb76fb-081d-48c8-8f38-23ebdd55e243 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.792161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceceeade-8be4-4a9b-94d0-edfd764cfee2 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.805008 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceceeade-8be4-4a9b-94d0-edfd764cfee2 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.811223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a026b-2c74-40d6-aed6-ffc5b15b21e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.812225 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a026b-2c74-40d6-aed6-ffc5b15b21e6 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.812518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a026b-2c74-40d6-aed6-ffc5b15b21e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.812804 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a026b-2c74-40d6-aed6-ffc5b15b21e6 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.813854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a026b-2c74-40d6-aed6-ffc5b15b21e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.815624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a026b-2c74-40d6-aed6-ffc5b15b21e6 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.820675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 817/1575] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/projects => generated 389 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.823007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb76fb-081d-48c8-8f38-23ebdd55e243 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.824324 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:53.843566 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 759/1576] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/users => generated 289 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.847508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a026b-2c74-40d6-aed6-ffc5b15b21e6 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.858771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a026b-2c74-40d6-aed6-ffc5b15b21e6 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.900465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb76fb-081d-48c8-8f38-23ebdd55e243 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.906350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bfbaf-840f-484c-896a-93dd7d2bd75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.907062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bfbaf-840f-484c-896a-93dd7d2bd75d admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.907346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bfbaf-840f-484c-896a-93dd7d2bd75d admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.907691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bfbaf-840f-484c-896a-93dd7d2bd75d admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.909026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bfbaf-840f-484c-896a-93dd7d2bd75d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.911180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bfbaf-840f-484c-896a-93dd7d2bd75d admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.927360 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6bfcc-3c67-4a28-84b3-46cd1a560652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.927707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6bfcc-3c67-4a28-84b3-46cd1a560652 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.927825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6bfcc-3c67-4a28-84b3-46cd1a560652 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.928033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6bfcc-3c67-4a28-84b3-46cd1a560652 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.928773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6bfcc-3c67-4a28-84b3-46cd1a560652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.930492 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6bfcc-3c67-4a28-84b3-46cd1a560652 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.939811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 818/1577] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.939811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bfbaf-840f-484c-896a-93dd7d2bd75d admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.949114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 760/1578] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:53.949839 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6bfcc-3c67-4a28-84b3-46cd1a560652 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.955414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bfbaf-840f-484c-896a-93dd7d2bd75d admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.960933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e361e93-15a7-470e-99b5-4345c1cb6a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.961208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6bfcc-3c67-4a28-84b3-46cd1a560652 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.961635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e361e93-15a7-470e-99b5-4345c1cb6a44 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.961925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e361e93-15a7-470e-99b5-4345c1cb6a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.962204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e361e93-15a7-470e-99b5-4345c1cb6a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.962930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e361e93-15a7-470e-99b5-4345c1cb6a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.964528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e361e93-15a7-470e-99b5-4345c1cb6a44 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.966535 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc7b4e-7a53-4a66-82f7-2b6933179a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.967188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc7b4e-7a53-4a66-82f7-2b6933179a57 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.967477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc7b4e-7a53-4a66-82f7-2b6933179a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.967746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc7b4e-7a53-4a66-82f7-2b6933179a57 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.968443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc7b4e-7a53-4a66-82f7-2b6933179a57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.970982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc7b4e-7a53-4a66-82f7-2b6933179a57 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:53.976955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 819/1579] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:53.976955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e361e93-15a7-470e-99b5-4345c1cb6a44 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.988246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e361e93-15a7-470e-99b5-4345c1cb6a44 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:53.992882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32682c-d582-41c6-b616-5fdc9d837a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:53.993562 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32682c-d582-41c6-b616-5fdc9d837a97 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:53.993849 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32682c-d582-41c6-b616-5fdc9d837a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:53.994125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32682c-d582-41c6-b616-5fdc9d837a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:53.994831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32682c-d582-41c6-b616-5fdc9d837a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:53.998752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 761/1580] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:53.998752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc7b4e-7a53-4a66-82f7-2b6933179a57 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:53.998985 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32682c-d582-41c6-b616-5fdc9d837a97 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.006900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc7b4e-7a53-4a66-82f7-2b6933179a57 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.011900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd83c4-7de2-44a9-983e-1718c01ee49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.012575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd83c4-7de2-44a9-983e-1718c01ee49b admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.012865 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd83c4-7de2-44a9-983e-1718c01ee49b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.013151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd83c4-7de2-44a9-983e-1718c01ee49b admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.013935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd83c4-7de2-44a9-983e-1718c01ee49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.016003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd83c4-7de2-44a9-983e-1718c01ee49b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.030004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 820/1581] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:48:53 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:54.030004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32682c-d582-41c6-b616-5fdc9d837a97 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.030296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 762/1582] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.030296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd83c4-7de2-44a9-983e-1718c01ee49b admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.040316 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd83c4-7de2-44a9-983e-1718c01ee49b admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.095806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32682c-d582-41c6-b616-5fdc9d837a97 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.100545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17ef0e-a11a-409f-a9a4-32715082e91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.101055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19c6ce-9c06-4d49-aba7-96d358ad22df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.101262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17ef0e-a11a-409f-a9a4-32715082e91f admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.101583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17ef0e-a11a-409f-a9a4-32715082e91f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.101748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19c6ce-9c06-4d49-aba7-96d358ad22df admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.101859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17ef0e-a11a-409f-a9a4-32715082e91f admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.102038 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19c6ce-9c06-4d49-aba7-96d358ad22df admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.102309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19c6ce-9c06-4d49-aba7-96d358ad22df admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.102581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17ef0e-a11a-409f-a9a4-32715082e91f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.103045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19c6ce-9c06-4d49-aba7-96d358ad22df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.104485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17ef0e-a11a-409f-a9a4-32715082e91f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.104613 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19c6ce-9c06-4d49-aba7-96d358ad22df admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.113535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:54.114565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 763/1583] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.115587 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19c6ce-9c06-4d49-aba7-96d358ad22df admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.127809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19c6ce-9c06-4d49-aba7-96d358ad22df admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.132010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 821/1584] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:48:54 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:54.132879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17ef0e-a11a-409f-a9a4-32715082e91f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.136301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5fd8a-a85b-469f-b8b1-403b786e3992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.136994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5fd8a-a85b-469f-b8b1-403b786e3992 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.137312 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5fd8a-a85b-469f-b8b1-403b786e3992 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.137621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5fd8a-a85b-469f-b8b1-403b786e3992 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.138364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5fd8a-a85b-469f-b8b1-403b786e3992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.140206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5fd8a-a85b-469f-b8b1-403b786e3992 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.144118 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17ef0e-a11a-409f-a9a4-32715082e91f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.150077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2f020-ae5a-44b5-980f-b1b75b1d288f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.151004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2f020-ae5a-44b5-980f-b1b75b1d288f admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.151911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2f020-ae5a-44b5-980f-b1b75b1d288f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.151911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2f020-ae5a-44b5-980f-b1b75b1d288f admin admin] PATH_INFO: `/v3/projects/6cc474e26dd8444b99645279c4d133bf/users/9fa96f51390e46e991c6e9afa385d2f7/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.154662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 764/1585] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.155503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5fd8a-a85b-469f-b8b1-403b786e3992 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.171612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2f020-ae5a-44b5-980f-b1b75b1d288f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc474e26dd8444b99645279c4d133bf, user_id=9fa96f51390e46e991c6e9afa385d2f7, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.172659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: WARNING py.warnings [None req-40f2f020-ae5a-44b5-980f-b1b75b1d288f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:48:54.172659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: warnings.warn(msg) Jul 19 20:48:54.172659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: Jul 19 20:48:54.174697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2f020-ae5a-44b5-980f-b1b75b1d288f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.175987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5fd8a-a85b-469f-b8b1-403b786e3992 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.181487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397e4da-7f41-483e-9a14-dd8baaf29577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.183129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397e4da-7f41-483e-9a14-dd8baaf29577 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.183647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397e4da-7f41-483e-9a14-dd8baaf29577 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.184184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397e4da-7f41-483e-9a14-dd8baaf29577 admin admin] PATH_INFO: `/v3/projects/26af6697fa9045cd9e7f41d5c2feb9cf/users/4d4f31197f3f4447b39ece90120c5449/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.193068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 822/1586] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:48:54 2022] PUT /identity/v3/projects/6cc474e26dd8444b99645279c4d133bf/users/9fa96f51390e46e991c6e9afa385d2f7/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:48:54.194783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2f020-ae5a-44b5-980f-b1b75b1d288f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.205809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397e4da-7f41-483e-9a14-dd8baaf29577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26af6697fa9045cd9e7f41d5c2feb9cf, user_id=4d4f31197f3f4447b39ece90120c5449, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.208421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397e4da-7f41-483e-9a14-dd8baaf29577 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.222714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 765/1587] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:48:54 2022] PUT /identity/v3/projects/26af6697fa9045cd9e7f41d5c2feb9cf/users/4d4f31197f3f4447b39ece90120c5449/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:48:54.223616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397e4da-7f41-483e-9a14-dd8baaf29577 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.243883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:54.273089 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2f020-ae5a-44b5-980f-b1b75b1d288f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.287408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397e4da-7f41-483e-9a14-dd8baaf29577 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.292120 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10145def-a12e-4bc3-8442-33080211ae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.292839 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10145def-a12e-4bc3-8442-33080211ae52 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.293119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10145def-a12e-4bc3-8442-33080211ae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.293417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10145def-a12e-4bc3-8442-33080211ae52 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.294109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10145def-a12e-4bc3-8442-33080211ae52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.296014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10145def-a12e-4bc3-8442-33080211ae52 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.316204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 766/1588] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:48:54 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:54.317023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10145def-a12e-4bc3-8442-33080211ae52 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.326453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10145def-a12e-4bc3-8442-33080211ae52 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.331828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f250e-ebb0-49b2-b022-4cf462831c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.332501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f250e-ebb0-49b2-b022-4cf462831c20 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.332794 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f250e-ebb0-49b2-b022-4cf462831c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.333104 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f250e-ebb0-49b2-b022-4cf462831c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.333863 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f250e-ebb0-49b2-b022-4cf462831c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.335074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d4a79-72e7-4c44-ba3e-d8b149c3839f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.335465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f250e-ebb0-49b2-b022-4cf462831c20 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.335746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d4a79-72e7-4c44-ba3e-d8b149c3839f admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.336039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d4a79-72e7-4c44-ba3e-d8b149c3839f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.336321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d4a79-72e7-4c44-ba3e-d8b149c3839f admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.337094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d4a79-72e7-4c44-ba3e-d8b149c3839f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.338711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d4a79-72e7-4c44-ba3e-d8b149c3839f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.346249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 767/1589] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.347065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f250e-ebb0-49b2-b022-4cf462831c20 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.351941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 823/1590] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.352816 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d4a79-72e7-4c44-ba3e-d8b149c3839f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.357237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f250e-ebb0-49b2-b022-4cf462831c20 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.362243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d3d03-d88c-47ce-88ca-175039453583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.362665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d4a79-72e7-4c44-ba3e-d8b149c3839f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.362902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d3d03-d88c-47ce-88ca-175039453583 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.363212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d3d03-d88c-47ce-88ca-175039453583 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.363487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d3d03-d88c-47ce-88ca-175039453583 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.364210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d3d03-d88c-47ce-88ca-175039453583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.365902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d3d03-d88c-47ce-88ca-175039453583 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.367284 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf805f1-087b-48ad-9dd9-5927bc3ec2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.367946 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf805f1-087b-48ad-9dd9-5927bc3ec2ca admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.368229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf805f1-087b-48ad-9dd9-5927bc3ec2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.368510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf805f1-087b-48ad-9dd9-5927bc3ec2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.369234 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf805f1-087b-48ad-9dd9-5927bc3ec2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.370857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf805f1-087b-48ad-9dd9-5927bc3ec2ca admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.382084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 824/1591] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.382836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 768/1592] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.383027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf805f1-087b-48ad-9dd9-5927bc3ec2ca admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.384227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade67cd0-9796-4492-9f23-d06dac498181 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.384632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade67cd0-9796-4492-9f23-d06dac498181 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.385068 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade67cd0-9796-4492-9f23-d06dac498181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.394426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf805f1-087b-48ad-9dd9-5927bc3ec2ca admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.399159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d5d58-5379-47ed-a61d-f56603bea4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.399856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d5d58-5379-47ed-a61d-f56603bea4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.400137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d5d58-5379-47ed-a61d-f56603bea4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.400412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d5d58-5379-47ed-a61d-f56603bea4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.401167 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d5d58-5379-47ed-a61d-f56603bea4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.402351 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ade67cd0-9796-4492-9f23-d06dac498181 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:54.402820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d5d58-5379-47ed-a61d-f56603bea4d0 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.403584 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ade67cd0-9796-4492-9f23-d06dac498181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:54.411454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 769/1593] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:54.412669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6526c12-e5b8-4792-9d75-718492933ae7 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.412961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6526c12-e5b8-4792-9d75-718492933ae7 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.413293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6526c12-e5b8-4792-9d75-718492933ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.416961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 825/1594] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.416961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d5d58-5379-47ed-a61d-f56603bea4d0 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.424549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d5d58-5379-47ed-a61d-f56603bea4d0 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.430585 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a474369-e6a8-4496-898f-28d5df6b2198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.432149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a474369-e6a8-4496-898f-28d5df6b2198 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.432824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a474369-e6a8-4496-898f-28d5df6b2198 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.433122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a474369-e6a8-4496-898f-28d5df6b2198 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.433908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a474369-e6a8-4496-898f-28d5df6b2198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.435562 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a474369-e6a8-4496-898f-28d5df6b2198 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.446001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-f6526c12-e5b8-4792-9d75-718492933ae7 None None] MFA Rules not processed for user `4d4f31197f3f4447b39ece90120c5449`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:54.446650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 826/1595] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.447261 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-f6526c12-e5b8-4792-9d75-718492933ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:54.447460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a474369-e6a8-4496-898f-28d5df6b2198 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.459530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a474369-e6a8-4496-898f-28d5df6b2198 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.464262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28bd30-7dac-4ee6-8eff-90fbf00330ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.465149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28bd30-7dac-4ee6-8eff-90fbf00330ed admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.465475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28bd30-7dac-4ee6-8eff-90fbf00330ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.465761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28bd30-7dac-4ee6-8eff-90fbf00330ed admin admin] PATH_INFO: `/v3/projects/5da6c42976154440bd0459beb12477f2/users/af989ff6dc024ff19023e6f633915ec3/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.480774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28bd30-7dac-4ee6-8eff-90fbf00330ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=5da6c42976154440bd0459beb12477f2, user_id=af989ff6dc024ff19023e6f633915ec3, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.483455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28bd30-7dac-4ee6-8eff-90fbf00330ed admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.498271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 827/1596] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:48:54 2022] PUT /identity/v3/projects/5da6c42976154440bd0459beb12477f2/users/af989ff6dc024ff19023e6f633915ec3/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:48:54.499154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28bd30-7dac-4ee6-8eff-90fbf00330ed admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.522850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 770/1597] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:54.522850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6526c12-e5b8-4792-9d75-718492933ae7 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.526320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:54.536124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6526c12-e5b8-4792-9d75-718492933ae7 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.593512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28bd30-7dac-4ee6-8eff-90fbf00330ed admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.598766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be507c05-a4a5-4242-8359-cfb87e102921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.599438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be507c05-a4a5-4242-8359-cfb87e102921 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.599720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be507c05-a4a5-4242-8359-cfb87e102921 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.599992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be507c05-a4a5-4242-8359-cfb87e102921 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.600716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be507c05-a4a5-4242-8359-cfb87e102921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.602337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be507c05-a4a5-4242-8359-cfb87e102921 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.612575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adac113-0055-4094-92b5-4b6ba79fbe5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.613260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adac113-0055-4094-92b5-4b6ba79fbe5f admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.613577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adac113-0055-4094-92b5-4b6ba79fbe5f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.613859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adac113-0055-4094-92b5-4b6ba79fbe5f admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.613966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 828/1598] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.614692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adac113-0055-4094-92b5-4b6ba79fbe5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.614839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be507c05-a4a5-4242-8359-cfb87e102921 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.616534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adac113-0055-4094-92b5-4b6ba79fbe5f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.625924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be507c05-a4a5-4242-8359-cfb87e102921 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.630678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 771/1599] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.630678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adac113-0055-4094-92b5-4b6ba79fbe5f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.631567 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382892d-32c8-4d83-a4d9-816f630e51bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.632522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382892d-32c8-4d83-a4d9-816f630e51bb admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.632807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382892d-32c8-4d83-a4d9-816f630e51bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.633087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382892d-32c8-4d83-a4d9-816f630e51bb admin admin] PATH_INFO: `/v3/projects/0efbb2e43cc74d63ad721e3971e7b181/users/20eed246b8e140688a628664f29e38a4/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.638252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adac113-0055-4094-92b5-4b6ba79fbe5f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.645060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf21fb-43f8-4dcd-9315-7a8b04e26a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.645960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf21fb-43f8-4dcd-9315-7a8b04e26a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.646235 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf21fb-43f8-4dcd-9315-7a8b04e26a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.646526 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf21fb-43f8-4dcd-9315-7a8b04e26a8f admin admin] PATH_INFO: `/v3/projects/d3f748e85ac34f358265448ab5167639/users/a3827dae00a1476cb4a5f519fd4cf21b/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.648541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382892d-32c8-4d83-a4d9-816f630e51bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efbb2e43cc74d63ad721e3971e7b181, user_id=20eed246b8e140688a628664f29e38a4, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.648976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:54.651945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382892d-32c8-4d83-a4d9-816f630e51bb admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.663751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf21fb-43f8-4dcd-9315-7a8b04e26a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f748e85ac34f358265448ab5167639, user_id=a3827dae00a1476cb4a5f519fd4cf21b, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.665480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 829/1600] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:48:54 2022] PUT /identity/v3/projects/0efbb2e43cc74d63ad721e3971e7b181/users/20eed246b8e140688a628664f29e38a4/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:48:54.666108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382892d-32c8-4d83-a4d9-816f630e51bb admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.666558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf21fb-43f8-4dcd-9315-7a8b04e26a8f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.680657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 772/1601] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:48:54 2022] PUT /identity/v3/projects/d3f748e85ac34f358265448ab5167639/users/a3827dae00a1476cb4a5f519fd4cf21b/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:48:54.681485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf21fb-43f8-4dcd-9315-7a8b04e26a8f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.731587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382892d-32c8-4d83-a4d9-816f630e51bb admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.743967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf21fb-43f8-4dcd-9315-7a8b04e26a8f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.748631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e70835-c801-4a78-b873-e0bde65fe918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.749311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e70835-c801-4a78-b873-e0bde65fe918 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.749623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e70835-c801-4a78-b873-e0bde65fe918 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.749925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e70835-c801-4a78-b873-e0bde65fe918 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.750636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e70835-c801-4a78-b873-e0bde65fe918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.752955 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e70835-c801-4a78-b873-e0bde65fe918 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.763456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 773/1602] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:54.764946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6978f7-84d2-4cd3-bcea-83576dba3ef6 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.765242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6978f7-84d2-4cd3-bcea-83576dba3ef6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.765550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6978f7-84d2-4cd3-bcea-83576dba3ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.783369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ba6978f7-84d2-4cd3-bcea-83576dba3ef6 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:54.783695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1613f-269e-4575-b8f4-bbdc44b623e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.784340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1613f-269e-4575-b8f4-bbdc44b623e7 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.784463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ba6978f7-84d2-4cd3-bcea-83576dba3ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:54.784658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1613f-269e-4575-b8f4-bbdc44b623e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.784935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1613f-269e-4575-b8f4-bbdc44b623e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.792298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 774/1603] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:54.793680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b81ab-c2a0-48cb-8e6b-f2e3ea9980a6 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.793972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b81ab-c2a0-48cb-8e6b-f2e3ea9980a6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.794253 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b81ab-c2a0-48cb-8e6b-f2e3ea9980a6 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.795685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 775/1604] 10.222.0.70 () {54 vars in 962 bytes} [Tue Jul 19 20:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:48:54.796437 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b81ab-c2a0-48cb-8e6b-f2e3ea9980a6 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.803701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1613f-269e-4575-b8f4-bbdc44b623e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.805138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1613f-269e-4575-b8f4-bbdc44b623e7 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.811077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b81ab-c2a0-48cb-8e6b-f2e3ea9980a6 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.815993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 830/1605] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:54.816629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1613f-269e-4575-b8f4-bbdc44b623e7 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.819041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfba38-4134-4e49-874a-16619852f351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.820050 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfba38-4134-4e49-874a-16619852f351 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.820394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfba38-4134-4e49-874a-16619852f351 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.820682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfba38-4134-4e49-874a-16619852f351 admin admin] PATH_INFO: `/v3/projects/d67b41835e4249bd94ce3441ad210efa/users/ca86b3a603364ad5b0105ac3fcec559e/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.826619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1613f-269e-4575-b8f4-bbdc44b623e7 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.832221 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e2efc-0e16-4b3c-93ff-1cba38541f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.833182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e2efc-0e16-4b3c-93ff-1cba38541f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.833488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e2efc-0e16-4b3c-93ff-1cba38541f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.833734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e2efc-0e16-4b3c-93ff-1cba38541f4d admin admin] PATH_INFO: `/v3/projects/8298856e2d28408296cc7e4deca84846/users/31790576f8734f75adbbd8e9c0d61721/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.839746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfba38-4134-4e49-874a-16619852f351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d67b41835e4249bd94ce3441ad210efa, user_id=ca86b3a603364ad5b0105ac3fcec559e, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.843654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfba38-4134-4e49-874a-16619852f351 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.851407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e2efc-0e16-4b3c-93ff-1cba38541f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8298856e2d28408296cc7e4deca84846, user_id=31790576f8734f75adbbd8e9c0d61721, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:54.854105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e2efc-0e16-4b3c-93ff-1cba38541f4d admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:54.865460 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 776/1606] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:48:54 2022] PUT /identity/v3/projects/d67b41835e4249bd94ce3441ad210efa/users/ca86b3a603364ad5b0105ac3fcec559e/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:48:54.867338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa3501-1cbe-424e-86aa-c25a4f9febe6 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.867682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa3501-1cbe-424e-86aa-c25a4f9febe6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.867969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa3501-1cbe-424e-86aa-c25a4f9febe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.879501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 831/1607] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:48:54 2022] PUT /identity/v3/projects/8298856e2d28408296cc7e4deca84846/users/31790576f8734f75adbbd8e9c0d61721/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:48:54.884623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62aa3e8-8861-45d3-88fc-4d7f1852417e None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.884915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62aa3e8-8861-45d3-88fc-4d7f1852417e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.885198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62aa3e8-8861-45d3-88fc-4d7f1852417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:54.888345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ccaa3501-1cbe-424e-86aa-c25a4f9febe6 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:54.889693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ccaa3501-1cbe-424e-86aa-c25a4f9febe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:54.917032 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e62aa3e8-8861-45d3-88fc-4d7f1852417e None None] MFA Rules not processed for user `a3827dae00a1476cb4a5f519fd4cf21b`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:54.918306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e62aa3e8-8861-45d3-88fc-4d7f1852417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:54.932258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:54.966737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 777/1608] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:54.968164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa3501-1cbe-424e-86aa-c25a4f9febe6 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:54.982518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa3501-1cbe-424e-86aa-c25a4f9febe6 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:54.989083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd714f8-5b4a-458a-ac97-f8a056c3cddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:54.990049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd714f8-5b4a-458a-ac97-f8a056c3cddb admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:54.990338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd714f8-5b4a-458a-ac97-f8a056c3cddb admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:54.990652 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd714f8-5b4a-458a-ac97-f8a056c3cddb admin admin] PATH_INFO: `/v3/projects/36212363399d488ab24e0557dc741925/users/bcb767c1dfac460bac1b2a7f27554ec5/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.003429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 832/1609] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:55.008675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0cf75-32db-4332-9c92-6b273a3ea03e None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.008675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0cf75-32db-4332-9c92-6b273a3ea03e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.008675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0cf75-32db-4332-9c92-6b273a3ea03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.013302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd714f8-5b4a-458a-ac97-f8a056c3cddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=36212363399d488ab24e0557dc741925, user_id=bcb767c1dfac460bac1b2a7f27554ec5, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:55.019858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd714f8-5b4a-458a-ac97-f8a056c3cddb admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:55.049159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 778/1610] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:48:54 2022] PUT /identity/v3/projects/36212363399d488ab24e0557dc741925/users/bcb767c1dfac460bac1b2a7f27554ec5/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:48:55.049159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd714f8-5b4a-458a-ac97-f8a056c3cddb admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:55.072751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-4ad0cf75-32db-4332-9c92-6b273a3ea03e None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:55.074263 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4ad0cf75-32db-4332-9c92-6b273a3ea03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:55.085172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:55.121317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd714f8-5b4a-458a-ac97-f8a056c3cddb admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:55.128388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23c0cc-a8e5-47aa-9a2d-ccb0afa1a144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:55.129549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23c0cc-a8e5-47aa-9a2d-ccb0afa1a144 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.130027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23c0cc-a8e5-47aa-9a2d-ccb0afa1a144 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.130915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23c0cc-a8e5-47aa-9a2d-ccb0afa1a144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.154917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 833/1611] 10.222.0.70 () {58 vars in 1059 bytes} [Tue Jul 19 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:55.155316 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512f5d1-d587-4341-ac29-e94df9c61305 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.166421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512f5d1-d587-4341-ac29-e94df9c61305 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.166811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512f5d1-d587-4341-ac29-e94df9c61305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.199404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-b512f5d1-d587-4341-ac29-e94df9c61305 None None] MFA Rules not processed for user `ca86b3a603364ad5b0105ac3fcec559e`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:55.200837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b512f5d1-d587-4341-ac29-e94df9c61305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:55.224441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23c0cc-a8e5-47aa-9a2d-ccb0afa1a144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:55.227036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23c0cc-a8e5-47aa-9a2d-ccb0afa1a144 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:55.242487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 779/1612] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:55.245320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58b52-af42-47fa-abbb-1a82a6f50ef7 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.245872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58b52-af42-47fa-abbb-1a82a6f50ef7 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.246335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58b52-af42-47fa-abbb-1a82a6f50ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.269860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-78f58b52-af42-47fa-abbb-1a82a6f50ef7 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:55.271364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-78f58b52-af42-47fa-abbb-1a82a6f50ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:55.294254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 834/1613] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:55.295168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512f5d1-d587-4341-ac29-e94df9c61305 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:55.299816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 780/1614] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:55.301469 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684b909-a4a5-47be-8394-e9d384c0af9d None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.302330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684b909-a4a5-47be-8394-e9d384c0af9d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.302583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684b909-a4a5-47be-8394-e9d384c0af9d None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.304377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 781/1615] 10.222.0.70 () {54 vars in 962 bytes} [Tue Jul 19 20:48:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:48:55.305592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724947b5-6a32-4f08-b489-c4967141317a None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.305981 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724947b5-6a32-4f08-b489-c4967141317a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.306360 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724947b5-6a32-4f08-b489-c4967141317a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.317924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512f5d1-d587-4341-ac29-e94df9c61305 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:55.323259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb17fb-6b11-412e-9348-d17ead1f5ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:55.324315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb17fb-6b11-412e-9348-d17ead1f5ebb service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.324806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb17fb-6b11-412e-9348-d17ead1f5ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.325294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb17fb-6b11-412e-9348-d17ead1f5ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.331074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-724947b5-6a32-4f08-b489-c4967141317a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:55.332722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-724947b5-6a32-4f08-b489-c4967141317a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:55.338130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb17fb-6b11-412e-9348-d17ead1f5ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:55.340042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb17fb-6b11-412e-9348-d17ead1f5ebb service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:55.344780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 782/1616] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:55.350374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724947b5-6a32-4f08-b489-c4967141317a None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:55.354222 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 835/1617] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:55.355448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb17fb-6b11-412e-9348-d17ead1f5ebb service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:55.367322 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb17fb-6b11-412e-9348-d17ead1f5ebb service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:55.367975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724947b5-6a32-4f08-b489-c4967141317a None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:55.373734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41288582-2ba1-4bb7-bb12-3f1955b2b409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:55.374768 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41288582-2ba1-4bb7-bb12-3f1955b2b409 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.375255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41288582-2ba1-4bb7-bb12-3f1955b2b409 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.375385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b28a2-527e-4112-8e82-4de7e4c3919a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:55.375809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41288582-2ba1-4bb7-bb12-3f1955b2b409 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.376205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b28a2-527e-4112-8e82-4de7e4c3919a service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.376650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b28a2-527e-4112-8e82-4de7e4c3919a service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.377609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41288582-2ba1-4bb7-bb12-3f1955b2b409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:55.377730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b28a2-527e-4112-8e82-4de7e4c3919a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.380936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41288582-2ba1-4bb7-bb12-3f1955b2b409 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:55.384982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:55.426479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 836/1618] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:48:55 2022] POST /identity/v3/projects => generated 393 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:55.432283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcf6a5-e514-4ea3-8763-8ec6e76ac901 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.432703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcf6a5-e514-4ea3-8763-8ec6e76ac901 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.433882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcf6a5-e514-4ea3-8763-8ec6e76ac901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.463021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a0fcf6a5-e514-4ea3-8763-8ec6e76ac901 None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:55.466539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a0fcf6a5-e514-4ea3-8763-8ec6e76ac901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:55.512337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b28a2-527e-4112-8e82-4de7e4c3919a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:55.516259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b28a2-527e-4112-8e82-4de7e4c3919a service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:55.531095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 783/1619] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:55.556223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b28a2-527e-4112-8e82-4de7e4c3919a service nova] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:55.579737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b28a2-527e-4112-8e82-4de7e4c3919a service nova] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:55.586831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 837/1620] 10.222.0.70 () {58 vars in 1059 bytes} [Tue Jul 19 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:55.587942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcf6a5-e514-4ea3-8763-8ec6e76ac901 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:55.675840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3f280-f5bb-4ca8-b9c8-1bdfeb2d1f20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:55.676819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3f280-f5bb-4ca8-b9c8-1bdfeb2d1f20 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.677133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3f280-f5bb-4ca8-b9c8-1bdfeb2d1f20 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.677838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3f280-f5bb-4ca8-b9c8-1bdfeb2d1f20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.689846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcf6a5-e514-4ea3-8763-8ec6e76ac901 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:55.690731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3f280-f5bb-4ca8-b9c8-1bdfeb2d1f20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:55.692710 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3f280-f5bb-4ca8-b9c8-1bdfeb2d1f20 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:55.695681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d332d7e-7c20-4483-87ef-9cba586197aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:55.696457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d332d7e-7c20-4483-87ef-9cba586197aa admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.696815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d332d7e-7c20-4483-87ef-9cba586197aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.697181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d332d7e-7c20-4483-87ef-9cba586197aa admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.698149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d332d7e-7c20-4483-87ef-9cba586197aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:55.700667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d332d7e-7c20-4483-87ef-9cba586197aa admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:55.704802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 784/1621] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:55.705799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3f280-f5bb-4ca8-b9c8-1bdfeb2d1f20 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:55.710921 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:55.723378 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3f280-f5bb-4ca8-b9c8-1bdfeb2d1f20 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:55.729488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482e734-3ce9-4fd9-bcc7-9406cd1d2f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:55.730262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482e734-3ce9-4fd9-bcc7-9406cd1d2f2a service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.730651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482e734-3ce9-4fd9-bcc7-9406cd1d2f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.731085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482e734-3ce9-4fd9-bcc7-9406cd1d2f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.742426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 838/1622] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:48:55 2022] POST /identity/v3/users => generated 303 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:48:55.743435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d332d7e-7c20-4483-87ef-9cba586197aa admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:55.744353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482e734-3ce9-4fd9-bcc7-9406cd1d2f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:55.747308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482e734-3ce9-4fd9-bcc7-9406cd1d2f2a service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:55.758135 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d332d7e-7c20-4483-87ef-9cba586197aa admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:55.761684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 785/1623] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:55.763565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482e734-3ce9-4fd9-bcc7-9406cd1d2f2a service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:55.766451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10b8b4-30cf-4256-92fc-bc50d5a4377b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:55.767398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10b8b4-30cf-4256-92fc-bc50d5a4377b service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.767823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10b8b4-30cf-4256-92fc-bc50d5a4377b service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.768572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10b8b4-30cf-4256-92fc-bc50d5a4377b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.778577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482e734-3ce9-4fd9-bcc7-9406cd1d2f2a service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:55.788978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b8422-b0db-4184-ab2b-c8e70f2b3587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:55.790160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b8422-b0db-4184-ab2b-c8e70f2b3587 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.790468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b8422-b0db-4184-ab2b-c8e70f2b3587 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.791054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b8422-b0db-4184-ab2b-c8e70f2b3587 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.793290 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b8422-b0db-4184-ab2b-c8e70f2b3587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:55.795463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b8422-b0db-4184-ab2b-c8e70f2b3587 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:55.813733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 786/1624] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:55.819436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b8422-b0db-4184-ab2b-c8e70f2b3587 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:55.833883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b8422-b0db-4184-ab2b-c8e70f2b3587 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:55.844665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f90778-abfb-4d92-b862-71fe27344b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:55.846765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90778-abfb-4d92-b862-71fe27344b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.847164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90778-abfb-4d92-b862-71fe27344b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.847736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90778-abfb-4d92-b862-71fe27344b16 admin admin] PATH_INFO: `/v3/projects/91f79ecfa53845d2b3d843d2ef684525/users/e5e3cb4cba1349679b764c9daa535c97/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.854538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10b8b4-30cf-4256-92fc-bc50d5a4377b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:55.856161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10b8b4-30cf-4256-92fc-bc50d5a4377b service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:55.870436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f90778-abfb-4d92-b862-71fe27344b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f79ecfa53845d2b3d843d2ef684525, user_id=e5e3cb4cba1349679b764c9daa535c97, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:55.875107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f90778-abfb-4d92-b862-71fe27344b16 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:55.878310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 839/1625] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:55.897464 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 787/1626] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:48:55 2022] PUT /identity/v3/projects/91f79ecfa53845d2b3d843d2ef684525/users/e5e3cb4cba1349679b764c9daa535c97/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:48:55.902441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10b8b4-30cf-4256-92fc-bc50d5a4377b service nova] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:55.910235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:55.919447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10b8b4-30cf-4256-92fc-bc50d5a4377b service nova] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:55.972768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c6096-89de-49b2-8fa7-a2250c824063 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.975681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c6096-89de-49b2-8fa7-a2250c824063 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.976632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c6096-89de-49b2-8fa7-a2250c824063 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:55.979629 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 788/1627] 10.222.0.70 () {54 vars in 912 bytes} [Tue Jul 19 20:48:55 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:48:55.987558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254b748-df55-4836-a58e-fcfe71b78bb5 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:55.987879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254b748-df55-4836-a58e-fcfe71b78bb5 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:55.988326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254b748-df55-4836-a58e-fcfe71b78bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:56.013392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3bab4-8a46-41e2-babf-0c64149bd41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:56.014476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3bab4-8a46-41e2-babf-0c64149bd41a admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:56.014826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3bab4-8a46-41e2-babf-0c64149bd41a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:56.015224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3bab4-8a46-41e2-babf-0c64149bd41a admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:56.015973 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3bab4-8a46-41e2-babf-0c64149bd41a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:56.018371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3bab4-8a46-41e2-babf-0c64149bd41a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:56.022059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c254b748-df55-4836-a58e-fcfe71b78bb5 None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:56.024001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c254b748-df55-4836-a58e-fcfe71b78bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:56.034613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 840/1628] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:56.035538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3bab4-8a46-41e2-babf-0c64149bd41a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:56.052968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3bab4-8a46-41e2-babf-0c64149bd41a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:56.058463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7054d4-47fb-4fcf-854e-23aaa5fe1b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:56.059379 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7054d4-47fb-4fcf-854e-23aaa5fe1b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:56.059661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7054d4-47fb-4fcf-854e-23aaa5fe1b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:56.059938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7054d4-47fb-4fcf-854e-23aaa5fe1b87 admin admin] PATH_INFO: `/v3/projects/91f79ecfa53845d2b3d843d2ef684525/users/e5e3cb4cba1349679b764c9daa535c97/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:56.061996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7054d4-47fb-4fcf-854e-23aaa5fe1b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f79ecfa53845d2b3d843d2ef684525, user_id=e5e3cb4cba1349679b764c9daa535c97, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:56.066906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7054d4-47fb-4fcf-854e-23aaa5fe1b87 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:56.102387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 841/1629] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:48:56 2022] PUT /identity/v3/projects/91f79ecfa53845d2b3d843d2ef684525/users/e5e3cb4cba1349679b764c9daa535c97/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:48:56.121415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1859cf7-cb50-4314-a592-33d8187b5e11 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:56.122006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1859cf7-cb50-4314-a592-33d8187b5e11 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:56.122547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1859cf7-cb50-4314-a592-33d8187b5e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:56.205018 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-b1859cf7-cb50-4314-a592-33d8187b5e11 None None] MFA Rules not processed for user `e5e3cb4cba1349679b764c9daa535c97`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:56.206615 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b1859cf7-cb50-4314-a592-33d8187b5e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:56.212117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 789/1630] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:48:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:56.226315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254b748-df55-4836-a58e-fcfe71b78bb5 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:56.338857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254b748-df55-4836-a58e-fcfe71b78bb5 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:56.400760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 842/1631] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:56.402465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a983888-9218-4bb2-abcb-7084c6a437b8 None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:56.402919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a983888-9218-4bb2-abcb-7084c6a437b8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:56.403295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a983888-9218-4bb2-abcb-7084c6a437b8 None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:56.405024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 843/1632] 10.222.0.70 () {54 vars in 912 bytes} [Tue Jul 19 20:48:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:48:56.413388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c7447-24b6-4c04-81a4-6bb3dc2678f1 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:56.413533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c7447-24b6-4c04-81a4-6bb3dc2678f1 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:56.413982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c7447-24b6-4c04-81a4-6bb3dc2678f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:56.444270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-279c7447-24b6-4c04-81a4-6bb3dc2678f1 None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:56.445644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ed6ab-bd44-40e2-bec7-f8be4e8dba7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:56.446713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ed6ab-bd44-40e2-bec7-f8be4e8dba7c service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:56.446915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-279c7447-24b6-4c04-81a4-6bb3dc2678f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:56.447652 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ed6ab-bd44-40e2-bec7-f8be4e8dba7c service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:56.448333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ed6ab-bd44-40e2-bec7-f8be4e8dba7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:56.480613 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:56.602051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ed6ab-bd44-40e2-bec7-f8be4e8dba7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:56.605261 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ed6ab-bd44-40e2-bec7-f8be4e8dba7c service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:56.619802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 844/1633] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:56.621082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c7447-24b6-4c04-81a4-6bb3dc2678f1 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:56.637664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:56.639204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 790/1634] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:48:56.646874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c7447-24b6-4c04-81a4-6bb3dc2678f1 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:56.654375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1db44-ef69-4a81-9b8c-82663d80d4e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:56.655665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1db44-ef69-4a81-9b8c-82663d80d4e2 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:56.656465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1db44-ef69-4a81-9b8c-82663d80d4e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:56.656945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1db44-ef69-4a81-9b8c-82663d80d4e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:56.672630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1db44-ef69-4a81-9b8c-82663d80d4e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:56.675050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1db44-ef69-4a81-9b8c-82663d80d4e2 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:56.690272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 845/1635] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:56.800491 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ed6ab-bd44-40e2-bec7-f8be4e8dba7c service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:56.811170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ed6ab-bd44-40e2-bec7-f8be4e8dba7c service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:56.817506 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df36e8c-e508-4f52-a2d9-353cb133ba08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:56.818586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df36e8c-e508-4f52-a2d9-353cb133ba08 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:56.819080 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df36e8c-e508-4f52-a2d9-353cb133ba08 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:56.819582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df36e8c-e508-4f52-a2d9-353cb133ba08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:56.861713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df36e8c-e508-4f52-a2d9-353cb133ba08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:56.863863 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df36e8c-e508-4f52-a2d9-353cb133ba08 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:56.911695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 791/1636] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:57.573221 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1db44-ef69-4a81-9b8c-82663d80d4e2 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:57.588784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1db44-ef69-4a81-9b8c-82663d80d4e2 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:57.610480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323e01e-621e-4952-95e7-aec6b5d41412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:57.618015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323e01e-621e-4952-95e7-aec6b5d41412 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:57.621527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323e01e-621e-4952-95e7-aec6b5d41412 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:57.621931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323e01e-621e-4952-95e7-aec6b5d41412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:57.789488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323e01e-621e-4952-95e7-aec6b5d41412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:57.791008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323e01e-621e-4952-95e7-aec6b5d41412 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:57.834062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 846/1637] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:59.750225 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94c0a5-cbdb-480d-a8da-05a6b7323dfb None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:59.750853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94c0a5-cbdb-480d-a8da-05a6b7323dfb None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:59.750989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94c0a5-cbdb-480d-a8da-05a6b7323dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:59.824351 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-9a94c0a5-cbdb-480d-a8da-05a6b7323dfb None None] MFA Rules not processed for user `bcb767c1dfac460bac1b2a7f27554ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:59.828547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-9a94c0a5-cbdb-480d-a8da-05a6b7323dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:00.067235 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 792/1638] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:48:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:00.068480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323e01e-621e-4952-95e7-aec6b5d41412 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:00.246373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323e01e-621e-4952-95e7-aec6b5d41412 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:00.254849 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53314687-9715-466b-a767-8080e711819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:00.256613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53314687-9715-466b-a767-8080e711819a admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:00.257173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53314687-9715-466b-a767-8080e711819a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:00.257850 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53314687-9715-466b-a767-8080e711819a admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:00.258754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53314687-9715-466b-a767-8080e711819a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:00.266245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53314687-9715-466b-a767-8080e711819a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:00.333635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94c0a5-cbdb-480d-a8da-05a6b7323dfb None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:00.339274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 847/1639] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:00 2022] POST /identity/v3/projects => generated 405 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:00.390901 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94c0a5-cbdb-480d-a8da-05a6b7323dfb None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:00.550739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfa671-243c-459c-bc7e-5f7f5cd722ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:00.551457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfa671-243c-459c-bc7e-5f7f5cd722ce admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:00.551927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfa671-243c-459c-bc7e-5f7f5cd722ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:00.554329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfa671-243c-459c-bc7e-5f7f5cd722ce admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:00.556028 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfa671-243c-459c-bc7e-5f7f5cd722ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:00.561047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfa671-243c-459c-bc7e-5f7f5cd722ce admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:00.613459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53314687-9715-466b-a767-8080e711819a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:00.620078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 793/1640] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:00 2022] POST /identity/v3/users => generated 309 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:00.651982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53314687-9715-466b-a767-8080e711819a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:00.664902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5206448-f0b4-4f9e-a3b0-c1705508b889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:00.666249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5206448-f0b4-4f9e-a3b0-c1705508b889 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:00.666758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5206448-f0b4-4f9e-a3b0-c1705508b889 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:00.667229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5206448-f0b4-4f9e-a3b0-c1705508b889 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:00.668166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5206448-f0b4-4f9e-a3b0-c1705508b889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:00.670583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5206448-f0b4-4f9e-a3b0-c1705508b889 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:00.693671 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 848/1641] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:00.694458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfa671-243c-459c-bc7e-5f7f5cd722ce admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:00.732167 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfa671-243c-459c-bc7e-5f7f5cd722ce admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:00.739486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51426f-82c3-443b-8ce1-f4cdcf902c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:00.740943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51426f-82c3-443b-8ce1-f4cdcf902c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:00.741474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51426f-82c3-443b-8ce1-f4cdcf902c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:00.742847 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51426f-82c3-443b-8ce1-f4cdcf902c22 admin admin] PATH_INFO: `/v3/projects/3c5d166b89bc4102aeadc9123f7e6c5e/users/7830748506414390909894ae75c623f8/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:00.747552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:00.807010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51426f-82c3-443b-8ce1-f4cdcf902c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5d166b89bc4102aeadc9123f7e6c5e, user_id=7830748506414390909894ae75c623f8, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:00.810070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51426f-82c3-443b-8ce1-f4cdcf902c22 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:00.835498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 794/1642] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:00 2022] PUT /identity/v3/projects/3c5d166b89bc4102aeadc9123f7e6c5e/users/7830748506414390909894ae75c623f8/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:00.836619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5206448-f0b4-4f9e-a3b0-c1705508b889 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:00.859390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5206448-f0b4-4f9e-a3b0-c1705508b889 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:01.081295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38a070-fa26-4ff9-af36-8677c25a5bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:01.083807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a070-fa26-4ff9-af36-8677c25a5bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:01.085154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a070-fa26-4ff9-af36-8677c25a5bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:01.087008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38a070-fa26-4ff9-af36-8677c25a5bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:01.090928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38a070-fa26-4ff9-af36-8677c25a5bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:01.095230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38a070-fa26-4ff9-af36-8677c25a5bc8 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:01.107877 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:01.125813 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 849/1643] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:01.127125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51426f-82c3-443b-8ce1-f4cdcf902c22 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:01.151656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51426f-82c3-443b-8ce1-f4cdcf902c22 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:01.158790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5d620-fea6-47da-9c1c-b52388ae80b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:01.159810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5d620-fea6-47da-9c1c-b52388ae80b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:01.160192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5d620-fea6-47da-9c1c-b52388ae80b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:01.160543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5d620-fea6-47da-9c1c-b52388ae80b9 admin admin] PATH_INFO: `/v3/projects/3c5d166b89bc4102aeadc9123f7e6c5e/users/7830748506414390909894ae75c623f8/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:01.163045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5d620-fea6-47da-9c1c-b52388ae80b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5d166b89bc4102aeadc9123f7e6c5e, user_id=7830748506414390909894ae75c623f8, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:01.166252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5d620-fea6-47da-9c1c-b52388ae80b9 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:01.203302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 795/1644] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:01 2022] PUT /identity/v3/projects/3c5d166b89bc4102aeadc9123f7e6c5e/users/7830748506414390909894ae75c623f8/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:04.264834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee244a-2dd0-4ede-b989-9b80eed3c334 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:04.265419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee244a-2dd0-4ede-b989-9b80eed3c334 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:04.265620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee244a-2dd0-4ede-b989-9b80eed3c334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:04.327140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0bee244a-2dd0-4ede-b989-9b80eed3c334 None None] MFA Rules not processed for user `7830748506414390909894ae75c623f8`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:04.328516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0bee244a-2dd0-4ede-b989-9b80eed3c334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:04.521237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 850/1645] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:04 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:06.184397 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bf233-6f2d-4b57-8c03-caa2a4a4c7bf None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:06.185322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bf233-6f2d-4b57-8c03-caa2a4a4c7bf None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:06.185322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bf233-6f2d-4b57-8c03-caa2a4a4c7bf None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:06.186654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 796/1646] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 20:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:49:06.235224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dbac7-4db7-49fb-84b7-dc9f6cf6a63b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:06.236731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dbac7-4db7-49fb-84b7-dc9f6cf6a63b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:06.237161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dbac7-4db7-49fb-84b7-dc9f6cf6a63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:06.278618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-542dbac7-4db7-49fb-84b7-dc9f6cf6a63b None None] MFA Rules not processed for user `f27871b39b9b4edaa512bd5b7467174a`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:06.279883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-542dbac7-4db7-49fb-84b7-dc9f6cf6a63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:06.410972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499bf3f-dc10-448d-9f69-13f47c395c14 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:06.411221 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499bf3f-dc10-448d-9f69-13f47c395c14 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:06.411532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499bf3f-dc10-448d-9f69-13f47c395c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:06.431663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 851/1647] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 20:49:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:06.452962 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-7499bf3f-dc10-448d-9f69-13f47c395c14 None None] MFA Rules not processed for user `20eed246b8e140688a628664f29e38a4`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:06.455102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7499bf3f-dc10-448d-9f69-13f47c395c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:06.457563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7e089-d150-4c38-bf83-f56601f0568f None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:06.458018 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7e089-d150-4c38-bf83-f56601f0568f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:06.458463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7e089-d150-4c38-bf83-f56601f0568f None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:06.460341 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 852/1648] 10.222.0.70 () {54 vars in 961 bytes} [Tue Jul 19 20:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:49:06.465323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a4fa0-a8bc-4885-ad7b-feb38850d4c9 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:06.465838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a4fa0-a8bc-4885-ad7b-feb38850d4c9 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:06.466258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a4fa0-a8bc-4885-ad7b-feb38850d4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:06.501257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e84a4fa0-a8bc-4885-ad7b-feb38850d4c9 None None] MFA Rules not processed for user `f27871b39b9b4edaa512bd5b7467174a`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:06.507977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e84a4fa0-a8bc-4885-ad7b-feb38850d4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:06.528682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 853/1649] 10.222.0.70 () {58 vars in 1058 bytes} [Tue Jul 19 20:49:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:06.538047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a4fa0-a8bc-4885-ad7b-feb38850d4c9 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:06.556747 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a4fa0-a8bc-4885-ad7b-feb38850d4c9 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:06.569008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70b0ce-3499-4a36-90c5-fe3406920770 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:06.569782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70b0ce-3499-4a36-90c5-fe3406920770 service placement] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:06.570067 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70b0ce-3499-4a36-90c5-fe3406920770 service placement] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:06.570343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70b0ce-3499-4a36-90c5-fe3406920770 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:06.594650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70b0ce-3499-4a36-90c5-fe3406920770 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:06.596980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70b0ce-3499-4a36-90c5-fe3406920770 service placement] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:06.620815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 854/1650] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:49:06.637048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 797/1651] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:06.651711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc09b93-7e93-4511-a9e0-1b9022da1596 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:06.652595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc09b93-7e93-4511-a9e0-1b9022da1596 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:06.653711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc09b93-7e93-4511-a9e0-1b9022da1596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:06.685324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-ccc09b93-7e93-4511-a9e0-1b9022da1596 None None] MFA Rules not processed for user `20eed246b8e140688a628664f29e38a4`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:06.688243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ccc09b93-7e93-4511-a9e0-1b9022da1596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:06.707965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 855/1652] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:06.719129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499bf3f-dc10-448d-9f69-13f47c395c14 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:06.818864 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499bf3f-dc10-448d-9f69-13f47c395c14 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:06.826612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b717e-a417-4d82-8bb5-8ba6cc796eea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:06.827403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b717e-a417-4d82-8bb5-8ba6cc796eea service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:06.827764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b717e-a417-4d82-8bb5-8ba6cc796eea service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:06.828116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b717e-a417-4d82-8bb5-8ba6cc796eea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:06.844175 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b717e-a417-4d82-8bb5-8ba6cc796eea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:06.846303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b717e-a417-4d82-8bb5-8ba6cc796eea service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:06.857085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 798/1653] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:07.519412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650174ce-5562-48cd-a1a7-7be3b1f36e0e None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:07.520095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650174ce-5562-48cd-a1a7-7be3b1f36e0e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:07.521869 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650174ce-5562-48cd-a1a7-7be3b1f36e0e None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:07.524180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 856/1654] 10.222.0.70 () {54 vars in 912 bytes} [Tue Jul 19 20:49:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:49:07.534248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f52a57-46d1-496c-b9da-16bcef660cef None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:07.535231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f52a57-46d1-496c-b9da-16bcef660cef None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:07.535762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f52a57-46d1-496c-b9da-16bcef660cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:07.598461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-80f52a57-46d1-496c-b9da-16bcef660cef None None] MFA Rules not processed for user `fb186ee2e3c94ca7ad203c9c0e32e498`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:07.600213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-80f52a57-46d1-496c-b9da-16bcef660cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:07.670599 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:07.764303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 799/1655] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:49:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:07.779595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650174ce-5562-48cd-a1a7-7be3b1f36e0e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:07.800414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650174ce-5562-48cd-a1a7-7be3b1f36e0e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:07.807402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93cec7-d7ed-491d-b9e3-4c7ffb7ca699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:07.808819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93cec7-d7ed-491d-b9e3-4c7ffb7ca699 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:07.809508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93cec7-d7ed-491d-b9e3-4c7ffb7ca699 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:07.809998 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93cec7-d7ed-491d-b9e3-4c7ffb7ca699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:07.827223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93cec7-d7ed-491d-b9e3-4c7ffb7ca699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:07.831584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93cec7-d7ed-491d-b9e3-4c7ffb7ca699 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:07.857834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 857/1656] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:49:08.014457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a965a-decd-40bd-8297-ec0d331ae31d None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:08.015031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a965a-decd-40bd-8297-ec0d331ae31d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:08.015476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a965a-decd-40bd-8297-ec0d331ae31d None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:08.023403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 800/1657] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 20:49:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:49:08.114549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93cec7-d7ed-491d-b9e3-4c7ffb7ca699 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:08.184206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbae87-f59b-42d1-b9b0-c44b1031d90e None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:08.184490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbae87-f59b-42d1-b9b0-c44b1031d90e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:08.184671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbae87-f59b-42d1-b9b0-c44b1031d90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:08.253932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-dedbae87-f59b-42d1-b9b0-c44b1031d90e None None] MFA Rules not processed for user `f27871b39b9b4edaa512bd5b7467174a`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:08.255191 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-dedbae87-f59b-42d1-b9b0-c44b1031d90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:08.309761 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 801/1658] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 20:49:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:08.311274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbae87-f59b-42d1-b9b0-c44b1031d90e None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:08.352838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:08.441789 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93cec7-d7ed-491d-b9e3-4c7ffb7ca699 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:08.470666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:08.471259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:08.471453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:08.471735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] PATH_INFO: `/v3/users/a3827dae00a1476cb4a5f519fd4cf21b` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:08.480186 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3827dae00a1476cb4a5f519fd4cf21b)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:08.481258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: WARNING py.warnings [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:49:08.481258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: warnings.warn(msg) Jul 19 20:49:08.481258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: Jul 19 20:49:08.487050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:08.522016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbae87-f59b-42d1-b9b0-c44b1031d90e None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:08.531291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:08.543671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:08.544259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:08.544349 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] PATH_INFO: `/v3/users/bcb767c1dfac460bac1b2a7f27554ec5` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:08.553009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb767c1dfac460bac1b2a7f27554ec5)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:08.561780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:08.806810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb767c1dfac460bac1b2a7f27554ec5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:08.820134 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3827dae00a1476cb4a5f519fd4cf21b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:08.828381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb767c1dfac460bac1b2a7f27554ec5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:08.845069 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb767c1dfac460bac1b2a7f27554ec5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:08.874238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3827dae00a1476cb4a5f519fd4cf21b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:08.888420 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb767c1dfac460bac1b2a7f27554ec5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:08.889749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb767c1dfac460bac1b2a7f27554ec5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:08.893028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3827dae00a1476cb4a5f519fd4cf21b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:08.893804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3827dae00a1476cb4a5f519fd4cf21b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:08.931077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 802/1659] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:08 2022] DELETE /identity/v3/users/bcb767c1dfac460bac1b2a7f27554ec5 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:08.931218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3827dae00a1476cb4a5f519fd4cf21b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:08.931361 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:08.935833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:08.956891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 858/1660] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:08 2022] DELETE /identity/v3/users/a3827dae00a1476cb4a5f519fd4cf21b => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:08.958592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:08.960874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:09.176105 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:09.202583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b4e49-9b3f-47be-969c-1cbb14a2906a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:09.217506 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a2a08-ed19-49f0-b2fd-1bef1d680f2e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:09.222911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c06ad5-a227-4e36-8603-350162411190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:09.224393 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06ad5-a227-4e36-8603-350162411190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:09.224874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06ad5-a227-4e36-8603-350162411190 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:09.225338 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06ad5-a227-4e36-8603-350162411190 admin admin] PATH_INFO: `/v3/users/ca86b3a603364ad5b0105ac3fcec559e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:09.230916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c06ad5-a227-4e36-8603-350162411190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca86b3a603364ad5b0105ac3fcec559e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:09.234751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c06ad5-a227-4e36-8603-350162411190 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:09.384552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-39c06ad5-a227-4e36-8603-350162411190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca86b3a603364ad5b0105ac3fcec559e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:09.427434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:09.433219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-39c06ad5-a227-4e36-8603-350162411190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca86b3a603364ad5b0105ac3fcec559e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:09.434197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cdbe4-e786-4b61-96a7-5614181d0b5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:09.435535 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cdbe4-e786-4b61-96a7-5614181d0b5e service placement] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:09.436232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cdbe4-e786-4b61-96a7-5614181d0b5e service placement] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:09.438607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cdbe4-e786-4b61-96a7-5614181d0b5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:09.450929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-597cdbe4-e786-4b61-96a7-5614181d0b5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:09.461278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-39c06ad5-a227-4e36-8603-350162411190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca86b3a603364ad5b0105ac3fcec559e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:09.463576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-39c06ad5-a227-4e36-8603-350162411190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca86b3a603364ad5b0105ac3fcec559e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:09.490763 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-39c06ad5-a227-4e36-8603-350162411190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca86b3a603364ad5b0105ac3fcec559e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:09.499354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cdbe4-e786-4b61-96a7-5614181d0b5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:09.501171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cdbe4-e786-4b61-96a7-5614181d0b5e service placement] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:09.515403 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 859/1661] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:08 2022] DELETE /identity/v3/users/ca86b3a603364ad5b0105ac3fcec559e => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:09.518385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbf322-5436-4519-8434-52b022defeb0 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:09.518688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbf322-5436-4519-8434-52b022defeb0 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:09.518978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbf322-5436-4519-8434-52b022defeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:09.533903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 803/1662] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:09.534969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cdbe4-e786-4b61-96a7-5614181d0b5e service placement] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:09.547165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-597cdbe4-e786-4b61-96a7-5614181d0b5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:09.580767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cdbe4-e786-4b61-96a7-5614181d0b5e service placement] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:09.589725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-22fbf322-5436-4519-8434-52b022defeb0 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:09.593016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-22fbf322-5436-4519-8434-52b022defeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:09.794700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 860/1663] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:09.796631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5665813-3dc4-4037-9f9d-45dfb990ed05 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:09.796930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5665813-3dc4-4037-9f9d-45dfb990ed05 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:09.797215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5665813-3dc4-4037-9f9d-45dfb990ed05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:09.851648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a5665813-3dc4-4037-9f9d-45dfb990ed05 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:09.853015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a5665813-3dc4-4037-9f9d-45dfb990ed05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:09.887017 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 861/1664] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:09.928100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b026a-0e92-4d32-9c00-82bf40f1da5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:09.928885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b026a-0e92-4d32-9c00-82bf40f1da5e service placement] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:09.929173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b026a-0e92-4d32-9c00-82bf40f1da5e service placement] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:09.929575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b026a-0e92-4d32-9c00-82bf40f1da5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:09.940490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-184b026a-0e92-4d32-9c00-82bf40f1da5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:09.960664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5665813-3dc4-4037-9f9d-45dfb990ed05 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:09.962234 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a5665813-3dc4-4037-9f9d-45dfb990ed05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:10.026928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b026a-0e92-4d32-9c00-82bf40f1da5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:10.028607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b026a-0e92-4d32-9c00-82bf40f1da5e service placement] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:10.092918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 804/1665] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:10.129622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5665813-3dc4-4037-9f9d-45dfb990ed05 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:10.139535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adac694-e4c3-4452-aecf-6dca630939a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:10.140739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adac694-e4c3-4452-aecf-6dca630939a7 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:10.141239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adac694-e4c3-4452-aecf-6dca630939a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:10.141897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adac694-e4c3-4452-aecf-6dca630939a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:10.156279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b026a-0e92-4d32-9c00-82bf40f1da5e service placement] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:10.165558 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adac694-e4c3-4452-aecf-6dca630939a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:10.168046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adac694-e4c3-4452-aecf-6dca630939a7 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:10.175852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b026a-0e92-4d32-9c00-82bf40f1da5e service placement] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:10.186053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 862/1666] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:10.190300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac34790-280f-4734-b6fd-3a8278dadc75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:10.205600 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac34790-280f-4734-b6fd-3a8278dadc75 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:10.206152 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac34790-280f-4734-b6fd-3a8278dadc75 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:10.206722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac34790-280f-4734-b6fd-3a8278dadc75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:10.235718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac34790-280f-4734-b6fd-3a8278dadc75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:10.241254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac34790-280f-4734-b6fd-3a8278dadc75 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:10.281451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 805/1667] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:49:11.516472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adac694-e4c3-4452-aecf-6dca630939a7 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:11.520617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6adac694-e4c3-4452-aecf-6dca630939a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:11.554226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adac694-e4c3-4452-aecf-6dca630939a7 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:11.561798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbcf3c-110e-4873-be76-6bb908f27c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:11.563172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcf3c-110e-4873-be76-6bb908f27c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:11.563732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcf3c-110e-4873-be76-6bb908f27c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:11.565030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcf3c-110e-4873-be76-6bb908f27c65 admin admin] PATH_INFO: `/v3/users/4d4f31197f3f4447b39ece90120c5449` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:11.568626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbcf3c-110e-4873-be76-6bb908f27c65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d4f31197f3f4447b39ece90120c5449)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:11.571324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbcf3c-110e-4873-be76-6bb908f27c65 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:11.659438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-bdfbcf3c-110e-4873-be76-6bb908f27c65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d4f31197f3f4447b39ece90120c5449'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:11.676645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-bdfbcf3c-110e-4873-be76-6bb908f27c65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d4f31197f3f4447b39ece90120c5449'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:11.699176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-bdfbcf3c-110e-4873-be76-6bb908f27c65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d4f31197f3f4447b39ece90120c5449'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:11.701634 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-bdfbcf3c-110e-4873-be76-6bb908f27c65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d4f31197f3f4447b39ece90120c5449'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:11.712398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:11.765884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-bdfbcf3c-110e-4873-be76-6bb908f27c65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d4f31197f3f4447b39ece90120c5449'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:11.786915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 863/1668] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:11 2022] DELETE /identity/v3/users/4d4f31197f3f4447b39ece90120c5449 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:11.791674 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bf72e-abec-4a41-99ae-7082921a9d9f None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:11.792209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bf72e-abec-4a41-99ae-7082921a9d9f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:11.792280 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bf72e-abec-4a41-99ae-7082921a9d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:11.830328 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-746bf72e-abec-4a41-99ae-7082921a9d9f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:11.833625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-746bf72e-abec-4a41-99ae-7082921a9d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:11.986858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 806/1669] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:11.991217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69f65-3573-43e3-b615-1fbf7de685e3 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:11.991529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69f65-3573-43e3-b615-1fbf7de685e3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:11.991816 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c69f65-3573-43e3-b615-1fbf7de685e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:11.992182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bf72e-abec-4a41-99ae-7082921a9d9f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:11.993907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-746bf72e-abec-4a41-99ae-7082921a9d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:12.006328 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:12.018309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bf72e-abec-4a41-99ae-7082921a9d9f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:12.027074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-75c69f65-3573-43e3-b615-1fbf7de685e3 None None] MFA Rules not processed for user `9fa96f51390e46e991c6e9afa385d2f7`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:12.028392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-75c69f65-3573-43e3-b615-1fbf7de685e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:12.033223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:12.034438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:12.034835 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:12.035221 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] PATH_INFO: `/v3/projects/36212363399d488ab24e0557dc741925` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:12.038950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] RBAC: Authorizing `identity:delete_project(project_id=36212363399d488ab24e0557dc741925)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:12.047143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: WARNING py.warnings [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:49:12.047143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: warnings.warn(msg) Jul 19 20:49:12.047143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: Jul 19 20:49:12.049714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:12.201756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36212363399d488ab24e0557dc741925', 'request_id': 'req-e5965fdb-bb55-4278-b952-c64def43596f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:12.210229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 864/1670] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:12.211250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c69f65-3573-43e3-b615-1fbf7de685e3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:12.214914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-75c69f65-3573-43e3-b615-1fbf7de685e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:12.259036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36212363399d488ab24e0557dc741925', 'request_id': 'req-e5965fdb-bb55-4278-b952-c64def43596f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:12.287148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36212363399d488ab24e0557dc741925', 'request_id': 'req-e5965fdb-bb55-4278-b952-c64def43596f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:12.310259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] The token cache is being invalidate because project 36212363399d488ab24e0557dc741925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:12.310790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:12.322496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e5965fdb-bb55-4278-b952-c64def43596f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:12.328326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 807/1671] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:11 2022] DELETE /identity/v3/projects/36212363399d488ab24e0557dc741925 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:12.331424 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819eac2-e4b8-4173-97bd-abcb097133ed None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:12.333878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819eac2-e4b8-4173-97bd-abcb097133ed None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:12.334572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819eac2-e4b8-4173-97bd-abcb097133ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:12.398936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c819eac2-e4b8-4173-97bd-abcb097133ed None None] MFA Rules not processed for user `9fa96f51390e46e991c6e9afa385d2f7`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:12.402691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c819eac2-e4b8-4173-97bd-abcb097133ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:12.412499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c69f65-3573-43e3-b615-1fbf7de685e3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:12.572177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 808/1672] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:12.573397 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819eac2-e4b8-4173-97bd-abcb097133ed None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:12.583907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c819eac2-e4b8-4173-97bd-abcb097133ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:12.624770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651434e-cc83-44d5-924b-538c5a1342da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:12.625533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651434e-cc83-44d5-924b-538c5a1342da service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:12.625825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651434e-cc83-44d5-924b-538c5a1342da service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:12.626110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651434e-cc83-44d5-924b-538c5a1342da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:12.637605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6651434e-cc83-44d5-924b-538c5a1342da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:12.755558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819eac2-e4b8-4173-97bd-abcb097133ed None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:12.771227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:12.772110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:12.772392 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:12.772711 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] PATH_INFO: `/v3/users/7830748506414390909894ae75c623f8` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:12.777822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:12.778405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7830748506414390909894ae75c623f8)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:12.780469 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:12.816276 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651434e-cc83-44d5-924b-538c5a1342da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:12.818097 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651434e-cc83-44d5-924b-538c5a1342da service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:12.858255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 865/1673] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:12.860852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651434e-cc83-44d5-924b-538c5a1342da service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:12.872974 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6651434e-cc83-44d5-924b-538c5a1342da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:12.896684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:12.920993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651434e-cc83-44d5-924b-538c5a1342da service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:12.940936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7830748506414390909894ae75c623f8'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:12.954739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:12.955996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:12.956544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:12.959373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] PATH_INFO: `/v3/projects/d3f748e85ac34f358265448ab5167639` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:12.960246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f748e85ac34f358265448ab5167639)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:12.962366 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:12.975037 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7830748506414390909894ae75c623f8'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:12.996734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7830748506414390909894ae75c623f8'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:13.035139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7830748506414390909894ae75c623f8'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:13.036254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7830748506414390909894ae75c623f8'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:13.052101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 809/1674] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:12 2022] DELETE /identity/v3/users/7830748506414390909894ae75c623f8 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:13.053216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:13.057311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:13.092153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f748e85ac34f358265448ab5167639', 'request_id': 'req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:13.107345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f748e85ac34f358265448ab5167639', 'request_id': 'req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:13.124081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f748e85ac34f358265448ab5167639', 'request_id': 'req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:13.154696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] The token cache is being invalidate because project d3f748e85ac34f358265448ab5167639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:13.155159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:13.162093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:13.167241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 866/1675] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:12 2022] DELETE /identity/v3/projects/d3f748e85ac34f358265448ab5167639 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:13.168329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:13.172758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:13.226797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2374d-3fa8-4a2e-bea9-1a36efa348e6 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:13.236002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb1033-a212-483b-b46c-eef3b0afe618 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:13.237265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb1033-a212-483b-b46c-eef3b0afe618 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:13.237733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb1033-a212-483b-b46c-eef3b0afe618 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:13.238227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb1033-a212-483b-b46c-eef3b0afe618 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:13.254386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-67fb1033-a212-483b-b46c-eef3b0afe618 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:13.436264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb1033-a212-483b-b46c-eef3b0afe618 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:13.438575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb1033-a212-483b-b46c-eef3b0afe618 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:13.472452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e91fce-8809-4bd7-a35e-fcfbf39189b2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:13.504318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 810/1676] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:13.506054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08cc1e-ac57-46ab-ae9e-547ecbe6d84d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:13.506356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08cc1e-ac57-46ab-ae9e-547ecbe6d84d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:13.506647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08cc1e-ac57-46ab-ae9e-547ecbe6d84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:13.623269 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0f08cc1e-ac57-46ab-ae9e-547ecbe6d84d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:13.630477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0f08cc1e-ac57-46ab-ae9e-547ecbe6d84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:13.762474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:13.763408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:13.763695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:13.763969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] PATH_INFO: `/v3/projects/d67b41835e4249bd94ce3441ad210efa` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:13.766252 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d67b41835e4249bd94ce3441ad210efa)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:13.782332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:13.813412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:13.821975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:13.868747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 811/1677] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:13.877948 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e82d2-a697-4e44-bf15-2b6b19bbc9d8 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:13.878419 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e82d2-a697-4e44-bf15-2b6b19bbc9d8 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:13.883048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e82d2-a697-4e44-bf15-2b6b19bbc9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:13.934017 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd67b41835e4249bd94ce3441ad210efa', 'request_id': 'req-938e2dcf-da02-4b96-b877-bc1d2eb587c3'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:13.945589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-fc9e82d2-a697-4e44-bf15-2b6b19bbc9d8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:13.950072 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-fc9e82d2-a697-4e44-bf15-2b6b19bbc9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:13.952966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd67b41835e4249bd94ce3441ad210efa', 'request_id': 'req-938e2dcf-da02-4b96-b877-bc1d2eb587c3'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:13.981921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd67b41835e4249bd94ce3441ad210efa', 'request_id': 'req-938e2dcf-da02-4b96-b877-bc1d2eb587c3'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:14.028171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] The token cache is being invalidate because project d67b41835e4249bd94ce3441ad210efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:14.028602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:14.030297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:14.035041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 867/1678] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:13 2022] DELETE /identity/v3/projects/d67b41835e4249bd94ce3441ad210efa => generated 0 bytes in 867 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:14.039564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:14.047786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:14.154078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 812/1679] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:14.155259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e82d2-a697-4e44-bf15-2b6b19bbc9d8 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:14.169601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e82d2-a697-4e44-bf15-2b6b19bbc9d8 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:14.196373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466c800-dd26-40a9-b361-6e40be2aac89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:14.198023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c800-dd26-40a9-b361-6e40be2aac89 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:14.199193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c800-dd26-40a9-b361-6e40be2aac89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:14.200153 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c800-dd26-40a9-b361-6e40be2aac89 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:14.202555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466c800-dd26-40a9-b361-6e40be2aac89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:14.202996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466c800-dd26-40a9-b361-6e40be2aac89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:14.206953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466c800-dd26-40a9-b361-6e40be2aac89 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:14.255013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 813/1680] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:14.258288 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466c800-dd26-40a9-b361-6e40be2aac89 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:14.283110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466c800-dd26-40a9-b361-6e40be2aac89 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:14.290560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877db14e-de43-456c-b1bf-ac0c2701582c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:14.291532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877db14e-de43-456c-b1bf-ac0c2701582c admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:14.293039 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877db14e-de43-456c-b1bf-ac0c2701582c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:14.293039 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877db14e-de43-456c-b1bf-ac0c2701582c admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:14.294161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877db14e-de43-456c-b1bf-ac0c2701582c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:14.297939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877db14e-de43-456c-b1bf-ac0c2701582c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:14.342745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 814/1681] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:14 2022] POST /identity/v3/projects => generated 405 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:14.348734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073998a0-1d26-45bc-a259-89bd658d3f53 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:14.349135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073998a0-1d26-45bc-a259-89bd658d3f53 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:14.349565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073998a0-1d26-45bc-a259-89bd658d3f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:14.367563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e2dcf-da02-4b96-b877-bc1d2eb587c3 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:14.427821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-073998a0-1d26-45bc-a259-89bd658d3f53 None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:14.429397 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-073998a0-1d26-45bc-a259-89bd658d3f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:14.537063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 815/1682] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:14.538291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073998a0-1d26-45bc-a259-89bd658d3f53 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:14.568280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:14.569493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:14.570004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:14.570589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] PATH_INFO: `/v3/users/e5e3cb4cba1349679b764c9daa535c97` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:14.572083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e3cb4cba1349679b764c9daa535c97)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:14.575984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:14.648898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:14.728395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e3cb4cba1349679b764c9daa535c97'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:14.752433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:14.777530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e3cb4cba1349679b764c9daa535c97'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:14.799043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073998a0-1d26-45bc-a259-89bd658d3f53 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:14.803405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e3cb4cba1349679b764c9daa535c97'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:14.804455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e3cb4cba1349679b764c9daa535c97'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:14.825108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670d438-859a-4352-bccb-cc4557dc7034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:14.825938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670d438-859a-4352-bccb-cc4557dc7034 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:14.826221 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670d438-859a-4352-bccb-cc4557dc7034 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:14.826504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670d438-859a-4352-bccb-cc4557dc7034 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:14.827230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670d438-859a-4352-bccb-cc4557dc7034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:14.833954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670d438-859a-4352-bccb-cc4557dc7034 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:14.834187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e3cb4cba1349679b764c9daa535c97'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:14.883647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 868/1683] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:14 2022] DELETE /identity/v3/users/e5e3cb4cba1349679b764c9daa535c97 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:14.885149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 816/1684] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:14 2022] POST /identity/v3/users => generated 303 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:14.889582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:14.889930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670d438-859a-4352-bccb-cc4557dc7034 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:14.896840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1670d438-859a-4352-bccb-cc4557dc7034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:14.900171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:14.937542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670d438-859a-4352-bccb-cc4557dc7034 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:15.134811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaf52b-81ab-4e57-b290-5e7dd5fda014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:15.136016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaf52b-81ab-4e57-b290-5e7dd5fda014 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:15.137160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaf52b-81ab-4e57-b290-5e7dd5fda014 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:15.137666 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaf52b-81ab-4e57-b290-5e7dd5fda014 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:15.138786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaf52b-81ab-4e57-b290-5e7dd5fda014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:15.142461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaf52b-81ab-4e57-b290-5e7dd5fda014 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:15.192932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 817/1685] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:15.194362 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaf52b-81ab-4e57-b290-5e7dd5fda014 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:15.217805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaf52b-81ab-4e57-b290-5e7dd5fda014 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:15.234460 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9305e5-e11c-4f4f-a44c-fcdbff387fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:15.235278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9305e5-e11c-4f4f-a44c-fcdbff387fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:15.235702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9305e5-e11c-4f4f-a44c-fcdbff387fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:15.237342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25187488-3d0b-49be-83ff-b2d07269d3e0 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:15.237627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9305e5-e11c-4f4f-a44c-fcdbff387fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:15.238504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9305e5-e11c-4f4f-a44c-fcdbff387fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:15.241094 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9305e5-e11c-4f4f-a44c-fcdbff387fc1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:15.258602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbdf45-35fc-40e6-a8f3-d4c56c526fb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:15.260110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbdf45-35fc-40e6-a8f3-d4c56c526fb9 service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:15.264493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbdf45-35fc-40e6-a8f3-d4c56c526fb9 service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:15.265183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbdf45-35fc-40e6-a8f3-d4c56c526fb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:15.271670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fdcbdf45-35fc-40e6-a8f3-d4c56c526fb9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:15.286645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 818/1686] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:15.293053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9305e5-e11c-4f4f-a44c-fcdbff387fc1 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:15.324461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9305e5-e11c-4f4f-a44c-fcdbff387fc1 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:15.342928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b6f32-db16-472b-8346-c51cbb16dd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:15.350542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b6f32-db16-472b-8346-c51cbb16dd79 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:15.351142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b6f32-db16-472b-8346-c51cbb16dd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:15.352357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b6f32-db16-472b-8346-c51cbb16dd79 admin admin] PATH_INFO: `/v3/projects/800423f34bdc4dbd81a9874217d3ceaf/users/c637cb6b8dfa4a308d90f35e9b7d0d92/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:15.410643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b6f32-db16-472b-8346-c51cbb16dd79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=800423f34bdc4dbd81a9874217d3ceaf, user_id=c637cb6b8dfa4a308d90f35e9b7d0d92, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:15.416321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b6f32-db16-472b-8346-c51cbb16dd79 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:15.477523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 819/1687] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:15 2022] PUT /identity/v3/projects/800423f34bdc4dbd81a9874217d3ceaf/users/c637cb6b8dfa4a308d90f35e9b7d0d92/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:15.494427 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fc94b-3cf6-46ff-a204-3e6e69f9cf8f None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:15.494816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fc94b-3cf6-46ff-a204-3e6e69f9cf8f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:15.495224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fc94b-3cf6-46ff-a204-3e6e69f9cf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:15.506110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbdf45-35fc-40e6-a8f3-d4c56c526fb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:15.510336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbdf45-35fc-40e6-a8f3-d4c56c526fb9 service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:15.543235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 869/1688] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:49:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:49:15.581767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-dc5fc94b-3cf6-46ff-a204-3e6e69f9cf8f None None] MFA Rules not processed for user `c637cb6b8dfa4a308d90f35e9b7d0d92`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:15.583817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-dc5fc94b-3cf6-46ff-a204-3e6e69f9cf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:15.638585 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbdf45-35fc-40e6-a8f3-d4c56c526fb9 service swift] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:15.642729 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fdcbdf45-35fc-40e6-a8f3-d4c56c526fb9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:15.721618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbdf45-35fc-40e6-a8f3-d4c56c526fb9 service swift] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:15.801644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 820/1689] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:15 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:15.802894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fc94b-3cf6-46ff-a204-3e6e69f9cf8f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:15.821230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:15.944028 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fc94b-3cf6-46ff-a204-3e6e69f9cf8f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:15.948925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b6b65-9fdf-4af1-a5ac-bb5ec518aaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:15.949540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b6b65-9fdf-4af1-a5ac-bb5ec518aaa8 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:15.949837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b6b65-9fdf-4af1-a5ac-bb5ec518aaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:15.950100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b6b65-9fdf-4af1-a5ac-bb5ec518aaa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:15.950811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b6b65-9fdf-4af1-a5ac-bb5ec518aaa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:15.952748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b6b65-9fdf-4af1-a5ac-bb5ec518aaa8 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:15.981523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:16.009597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 821/1690] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:15 2022] POST /identity/v3/projects => generated 403 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:49:16.014291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b6b65-9fdf-4af1-a5ac-bb5ec518aaa8 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:16.037394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:16.037394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:16.037394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:16.037394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] PATH_INFO: `/v3/projects/3c5d166b89bc4102aeadc9123f7e6c5e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:16.037855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c5d166b89bc4102aeadc9123f7e6c5e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:16.045832 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:16.181599 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b6b65-9fdf-4af1-a5ac-bb5ec518aaa8 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:16.197638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2439c-ec26-483d-9c43-f3e087c75ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:16.198279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2439c-ec26-483d-9c43-f3e087c75ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:16.198563 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2439c-ec26-483d-9c43-f3e087c75ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:16.198846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2439c-ec26-483d-9c43-f3e087c75ca2 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:16.199572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2439c-ec26-483d-9c43-f3e087c75ca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:16.212053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2439c-ec26-483d-9c43-f3e087c75ca2 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:16.264421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 822/1691] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:16 2022] POST /identity/v3/users => generated 308 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:16.265952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee81b1-4077-4a9b-bb36-85833160b95d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:16.266285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee81b1-4077-4a9b-bb36-85833160b95d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:16.266515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee81b1-4077-4a9b-bb36-85833160b95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:16.282079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c5d166b89bc4102aeadc9123f7e6c5e', 'request_id': 'req-9b807504-4ce1-4b74-9037-a94854447e62'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:16.303435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c5d166b89bc4102aeadc9123f7e6c5e', 'request_id': 'req-9b807504-4ce1-4b74-9037-a94854447e62'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:16.319576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c5d166b89bc4102aeadc9123f7e6c5e', 'request_id': 'req-9b807504-4ce1-4b74-9037-a94854447e62'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:16.356347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] The token cache is being invalidate because project 3c5d166b89bc4102aeadc9123f7e6c5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:16.356347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:16.356347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:16.359612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-d4ee81b1-4077-4a9b-bb36-85833160b95d None None] MFA Rules not processed for user `af989ff6dc024ff19023e6f633915ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:16.361556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d4ee81b1-4077-4a9b-bb36-85833160b95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:16.362174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 870/1692] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:15 2022] DELETE /identity/v3/projects/3c5d166b89bc4102aeadc9123f7e6c5e => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:16.363097 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:16.378022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:16.674192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 823/1693] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:16.685349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b807504-4ce1-4b74-9037-a94854447e62 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:16.687516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae8b67-cdbc-423a-a715-62a1b44aa72d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:16.687805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae8b67-cdbc-423a-a715-62a1b44aa72d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:16.688082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae8b67-cdbc-423a-a715-62a1b44aa72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:16.709528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186aafb-910b-49a0-a87d-7ed0a0b19b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:16.710147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186aafb-910b-49a0-a87d-7ed0a0b19b56 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:16.710433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186aafb-910b-49a0-a87d-7ed0a0b19b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:16.710717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186aafb-910b-49a0-a87d-7ed0a0b19b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:16.711485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186aafb-910b-49a0-a87d-7ed0a0b19b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:16.723629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186aafb-910b-49a0-a87d-7ed0a0b19b56 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:16.757114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 871/1694] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:16.765920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-9cae8b67-cdbc-423a-a715-62a1b44aa72d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:16.767207 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-9cae8b67-cdbc-423a-a715-62a1b44aa72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:16.769064 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186aafb-910b-49a0-a87d-7ed0a0b19b56 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:16.791095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 824/1695] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:16.792459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186aafb-910b-49a0-a87d-7ed0a0b19b56 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:16.796712 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f8340-5366-4543-8205-0a2b5fd084fe None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:16.797017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f8340-5366-4543-8205-0a2b5fd084fe None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:16.797274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f8340-5366-4543-8205-0a2b5fd084fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:16.807756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9636e2-53ed-403a-a5fe-6f25def30968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:16.808682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9636e2-53ed-403a-a5fe-6f25def30968 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:16.808970 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9636e2-53ed-403a-a5fe-6f25def30968 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:16.809255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9636e2-53ed-403a-a5fe-6f25def30968 admin admin] PATH_INFO: `/v3/projects/8e9749c8ff454aa69d9e11962973e95d/users/2a597573c6214add96f2222515ad15a2/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:16.814734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:16.839450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-187f8340-5366-4543-8205-0a2b5fd084fe None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:16.840710 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-187f8340-5366-4543-8205-0a2b5fd084fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:16.844116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9636e2-53ed-403a-a5fe-6f25def30968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9749c8ff454aa69d9e11962973e95d, user_id=2a597573c6214add96f2222515ad15a2, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:16.854882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9636e2-53ed-403a-a5fe-6f25def30968 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:16.872986 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 825/1696] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:16.878098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f8340-5366-4543-8205-0a2b5fd084fe None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:16.896175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 872/1697] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:16 2022] PUT /identity/v3/projects/8e9749c8ff454aa69d9e11962973e95d/users/2a597573c6214add96f2222515ad15a2/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:16.900185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9636e2-53ed-403a-a5fe-6f25def30968 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:16.905056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f8340-5366-4543-8205-0a2b5fd084fe None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:17.093169 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:17.100367 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d782bf-801a-45c7-b9de-f534e59a8eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:17.105394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d782bf-801a-45c7-b9de-f534e59a8eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:17.105394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d782bf-801a-45c7-b9de-f534e59a8eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:17.105394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d782bf-801a-45c7-b9de-f534e59a8eb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:17.105394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d782bf-801a-45c7-b9de-f534e59a8eb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:17.105394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d782bf-801a-45c7-b9de-f534e59a8eb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:17.105394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d782bf-801a-45c7-b9de-f534e59a8eb1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:17.113486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9636e2-53ed-403a-a5fe-6f25def30968 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:17.125786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6568-62a6-40e5-9f69-efc319deb404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:17.132010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6568-62a6-40e5-9f69-efc319deb404 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:17.132010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6568-62a6-40e5-9f69-efc319deb404 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:17.132010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6568-62a6-40e5-9f69-efc319deb404 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:17.139283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6568-62a6-40e5-9f69-efc319deb404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:17.141219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe6568-62a6-40e5-9f69-efc319deb404 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:17.158636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 826/1698] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:17.161733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d782bf-801a-45c7-b9de-f534e59a8eb1 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:17.166231 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 873/1699] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:17.167220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-85d782bf-801a-45c7-b9de-f534e59a8eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:17.169458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6568-62a6-40e5-9f69-efc319deb404 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:17.205594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6568-62a6-40e5-9f69-efc319deb404 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:17.242278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df816-4ca9-4569-abb1-1db4f54324ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:17.258030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df816-4ca9-4569-abb1-1db4f54324ae admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:17.258030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df816-4ca9-4569-abb1-1db4f54324ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:17.258030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df816-4ca9-4569-abb1-1db4f54324ae admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:17.258030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1df816-4ca9-4569-abb1-1db4f54324ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:17.265156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1df816-4ca9-4569-abb1-1db4f54324ae admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:17.374408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 874/1700] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:17 2022] POST /identity/v3/projects => generated 365 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:17.374408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df816-4ca9-4569-abb1-1db4f54324ae admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:17.441870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d782bf-801a-45c7-b9de-f534e59a8eb1 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:17.694439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df816-4ca9-4569-abb1-1db4f54324ae admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:17.716729 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e5775-65c0-48a9-968a-09d1f4f87b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:17.717950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e5775-65c0-48a9-968a-09d1f4f87b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:17.724602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e5775-65c0-48a9-968a-09d1f4f87b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:17.724914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e5775-65c0-48a9-968a-09d1f4f87b26 admin admin] PATH_INFO: `/v3/projects/8e9749c8ff454aa69d9e11962973e95d/users/2a597573c6214add96f2222515ad15a2/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:17.728746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e5775-65c0-48a9-968a-09d1f4f87b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9749c8ff454aa69d9e11962973e95d, user_id=2a597573c6214add96f2222515ad15a2, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:17.731156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e5775-65c0-48a9-968a-09d1f4f87b26 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:17.731681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf523d4-b02e-491c-aa08-b4b83bbb4c29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:17.733022 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf523d4-b02e-491c-aa08-b4b83bbb4c29 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:17.733556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf523d4-b02e-491c-aa08-b4b83bbb4c29 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:17.735509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf523d4-b02e-491c-aa08-b4b83bbb4c29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:17.793650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 875/1701] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:17 2022] PUT /identity/v3/projects/8e9749c8ff454aa69d9e11962973e95d/users/2a597573c6214add96f2222515ad15a2/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:17.793650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5035bb-2af9-420d-aafc-fd7f4684d11c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:17.793650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5035bb-2af9-420d-aafc-fd7f4684d11c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:17.793650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5035bb-2af9-420d-aafc-fd7f4684d11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:17.849592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:17.918173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-4a5035bb-2af9-420d-aafc-fd7f4684d11c None None] MFA Rules not processed for user `31790576f8734f75adbbd8e9c0d61721`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:17.919434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4a5035bb-2af9-420d-aafc-fd7f4684d11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:17.971423 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf523d4-b02e-491c-aa08-b4b83bbb4c29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:17.981314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf523d4-b02e-491c-aa08-b4b83bbb4c29 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:18.040878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 827/1702] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 878 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:18.042162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf523d4-b02e-491c-aa08-b4b83bbb4c29 service nova] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:18.068859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf523d4-b02e-491c-aa08-b4b83bbb4c29 service nova] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:18.171476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 876/1703] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:18.174991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5035bb-2af9-420d-aafc-fd7f4684d11c None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:18.197894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4a5035bb-2af9-420d-aafc-fd7f4684d11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:18.237065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99040aca-ab10-499b-9182-3aa7ce5a8535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:18.239174 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99040aca-ab10-499b-9182-3aa7ce5a8535 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:18.239748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99040aca-ab10-499b-9182-3aa7ce5a8535 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:18.240436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99040aca-ab10-499b-9182-3aa7ce5a8535 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:18.241402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99040aca-ab10-499b-9182-3aa7ce5a8535 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:18.244276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99040aca-ab10-499b-9182-3aa7ce5a8535 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:18.312591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 828/1704] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:18 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:18.314374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b9197-f133-442f-9acd-2e5f9c2c0e8d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:18.314890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b9197-f133-442f-9acd-2e5f9c2c0e8d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:18.315375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b9197-f133-442f-9acd-2e5f9c2c0e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:18.362095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ec7b9197-f133-442f-9acd-2e5f9c2c0e8d None None] MFA Rules not processed for user `9fa96f51390e46e991c6e9afa385d2f7`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:18.370651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ec7b9197-f133-442f-9acd-2e5f9c2c0e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:18.467423 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:18.508886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5035bb-2af9-420d-aafc-fd7f4684d11c None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:18.527759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:18.528901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:18.530991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:18.533883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] PATH_INFO: `/v3/projects/26af6697fa9045cd9e7f41d5c2feb9cf` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:18.542363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26af6697fa9045cd9e7f41d5c2feb9cf)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:18.546263 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:18.615068 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 829/1705] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:18.616607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1e6ac-8461-494e-9642-0b162cee70fe None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:18.619226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1e6ac-8461-494e-9642-0b162cee70fe None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:18.619604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1e6ac-8461-494e-9642-0b162cee70fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:18.716679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26af6697fa9045cd9e7f41d5c2feb9cf', 'request_id': 'req-b424335e-dba1-40e5-b20f-6efdde237c11'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:18.732953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-29f1e6ac-8461-494e-9642-0b162cee70fe None None] MFA Rules not processed for user `2a597573c6214add96f2222515ad15a2`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:18.734459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-29f1e6ac-8461-494e-9642-0b162cee70fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:18.740765 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26af6697fa9045cd9e7f41d5c2feb9cf', 'request_id': 'req-b424335e-dba1-40e5-b20f-6efdde237c11'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:18.789316 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26af6697fa9045cd9e7f41d5c2feb9cf', 'request_id': 'req-b424335e-dba1-40e5-b20f-6efdde237c11'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:18.866814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] The token cache is being invalidate because project 26af6697fa9045cd9e7f41d5c2feb9cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:18.867726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:18.872247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:18.888954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 877/1706] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:18 2022] DELETE /identity/v3/projects/26af6697fa9045cd9e7f41d5c2feb9cf => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:18.890324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:18.898504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:18.926279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:18.997775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 830/1707] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:19.004828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1e6ac-8461-494e-9642-0b162cee70fe None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:19.006517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-29f1e6ac-8461-494e-9642-0b162cee70fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:19.153996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1e6ac-8461-494e-9642-0b162cee70fe None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:19.160317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:19.162752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:19.163252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:19.163722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] PATH_INFO: `/v3/projects/91f79ecfa53845d2b3d843d2ef684525` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:19.165829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f79ecfa53845d2b3d843d2ef684525)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:19.168160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:19.331780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f79ecfa53845d2b3d843d2ef684525', 'request_id': 'req-96664a7a-fb81-440f-a437-e7b613ee6fb3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:19.356603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424335e-dba1-40e5-b20f-6efdde237c11 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:19.366158 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f79ecfa53845d2b3d843d2ef684525', 'request_id': 'req-96664a7a-fb81-440f-a437-e7b613ee6fb3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:19.388440 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f79ecfa53845d2b3d843d2ef684525', 'request_id': 'req-96664a7a-fb81-440f-a437-e7b613ee6fb3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:19.434148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] The token cache is being invalidate because project 91f79ecfa53845d2b3d843d2ef684525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:19.434579 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:19.436213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:19.442146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 831/1708] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:19 2022] DELETE /identity/v3/projects/91f79ecfa53845d2b3d843d2ef684525 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:19.447640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:19.458287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:19.605160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bed0b-fee8-4106-9241-b6c5bcfa4121 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:19.608293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bed0b-fee8-4106-9241-b6c5bcfa4121 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:19.609022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bed0b-fee8-4106-9241-b6c5bcfa4121 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:19.609587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bed0b-fee8-4106-9241-b6c5bcfa4121 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:19.614200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2c1bed0b-fee8-4106-9241-b6c5bcfa4121 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:19.658011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:19.717665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bed0b-fee8-4106-9241-b6c5bcfa4121 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:19.720911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bed0b-fee8-4106-9241-b6c5bcfa4121 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:19.731455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 878/1709] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 841 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:19.732863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90d73d-a887-4d12-b742-d3b28ec94862 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:19.733181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90d73d-a887-4d12-b742-d3b28ec94862 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:19.733483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90d73d-a887-4d12-b742-d3b28ec94862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:19.739979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96664a7a-fb81-440f-a437-e7b613ee6fb3 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:19.756510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-2e90d73d-a887-4d12-b742-d3b28ec94862 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:19.757825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2e90d73d-a887-4d12-b742-d3b28ec94862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:19.771511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020099c7-5123-4b2a-80aa-61343bc7bd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:19.773337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020099c7-5123-4b2a-80aa-61343bc7bd7d admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:19.773727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020099c7-5123-4b2a-80aa-61343bc7bd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:19.774291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020099c7-5123-4b2a-80aa-61343bc7bd7d admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:19.775908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 879/1710] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:19.776912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90d73d-a887-4d12-b742-d3b28ec94862 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:19.777040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020099c7-5123-4b2a-80aa-61343bc7bd7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:19.778493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2e90d73d-a887-4d12-b742-d3b28ec94862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:19.791190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020099c7-5123-4b2a-80aa-61343bc7bd7d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:19.841274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 832/1711] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:19.843278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0a4c-8621-4507-917b-49406c27cd40 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:19.843278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0a4c-8621-4507-917b-49406c27cd40 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:19.843617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0a4c-8621-4507-917b-49406c27cd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:19.924061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-234c0a4c-8621-4507-917b-49406c27cd40 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:19.925540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-234c0a4c-8621-4507-917b-49406c27cd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:19.946761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90d73d-a887-4d12-b742-d3b28ec94862 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:19.955838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5b69d-a211-4f1b-acb8-531039f8b782 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:19.956508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5b69d-a211-4f1b-acb8-531039f8b782 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:19.956806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5b69d-a211-4f1b-acb8-531039f8b782 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:19.957089 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5b69d-a211-4f1b-acb8-531039f8b782 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:19.957765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 833/1712] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:19.960797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3aa5b69d-a211-4f1b-acb8-531039f8b782 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:19.971896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afcae7-24ae-42bb-b2ba-b255b17bb25c None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:19.974000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afcae7-24ae-42bb-b2ba-b255b17bb25c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:19.974465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afcae7-24ae-42bb-b2ba-b255b17bb25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:20.007974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:20.045192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-b9afcae7-24ae-42bb-b2ba-b255b17bb25c None None] MFA Rules not processed for user `fb186ee2e3c94ca7ad203c9c0e32e498`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:20.046479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b9afcae7-24ae-42bb-b2ba-b255b17bb25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:20.152765 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5b69d-a211-4f1b-acb8-531039f8b782 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:20.154531 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5b69d-a211-4f1b-acb8-531039f8b782 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:20.175705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 880/1713] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:19 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:20.177075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29344278-f656-4505-99f1-85d3e7a9e0cb None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:20.177688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29344278-f656-4505-99f1-85d3e7a9e0cb None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:20.178084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29344278-f656-4505-99f1-85d3e7a9e0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:20.225865 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-29344278-f656-4505-99f1-85d3e7a9e0cb None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:20.227993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-29344278-f656-4505-99f1-85d3e7a9e0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:20.245498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 834/1715] 10.222.0.84 () {58 vars in 1015 bytes} [Tue Jul 19 20:49:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:20.245668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 881/1715] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:20.247261 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804bfef-b5ea-4154-a97c-20eb79fa800d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:20.247712 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804bfef-b5ea-4154-a97c-20eb79fa800d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:20.248138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804bfef-b5ea-4154-a97c-20eb79fa800d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:20.251182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29344278-f656-4505-99f1-85d3e7a9e0cb None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:20.277038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29344278-f656-4505-99f1-85d3e7a9e0cb None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:20.297056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427ff65-ba0c-421a-85ef-be8dfa89398e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:20.297976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427ff65-ba0c-421a-85ef-be8dfa89398e admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:20.298895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427ff65-ba0c-421a-85ef-be8dfa89398e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:20.299908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427ff65-ba0c-421a-85ef-be8dfa89398e admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:20.300342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427ff65-ba0c-421a-85ef-be8dfa89398e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:20.302137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427ff65-ba0c-421a-85ef-be8dfa89398e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:20.322552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-b804bfef-b5ea-4154-a97c-20eb79fa800d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:20.323106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 882/1716] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:20.324068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427ff65-ba0c-421a-85ef-be8dfa89398e admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:20.324403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b804bfef-b5ea-4154-a97c-20eb79fa800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:20.336152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427ff65-ba0c-421a-85ef-be8dfa89398e admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:20.350254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 835/1717] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:20.364362 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804bfef-b5ea-4154-a97c-20eb79fa800d None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:20.365509 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99443f73-a1c3-4721-ad0f-b12b646afa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:20.366806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99443f73-a1c3-4721-ad0f-b12b646afa6b admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:20.367456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99443f73-a1c3-4721-ad0f-b12b646afa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:20.367847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99443f73-a1c3-4721-ad0f-b12b646afa6b admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:20.368726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99443f73-a1c3-4721-ad0f-b12b646afa6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:20.369157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99443f73-a1c3-4721-ad0f-b12b646afa6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:20.370920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99443f73-a1c3-4721-ad0f-b12b646afa6b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:20.371605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b804bfef-b5ea-4154-a97c-20eb79fa800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:20.388072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 883/1718] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:20.394336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99443f73-a1c3-4721-ad0f-b12b646afa6b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:20.418835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99443f73-a1c3-4721-ad0f-b12b646afa6b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:20.423828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804bfef-b5ea-4154-a97c-20eb79fa800d None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:20.431732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daa523-e65a-4429-b6ea-eedcbdd6e3b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:20.432080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1720ed-409c-4840-b617-4b0406c113f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:20.432761 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daa523-e65a-4429-b6ea-eedcbdd6e3b5 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:20.433179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daa523-e65a-4429-b6ea-eedcbdd6e3b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:20.433607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daa523-e65a-4429-b6ea-eedcbdd6e3b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:20.437528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1720ed-409c-4840-b617-4b0406c113f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:20.437826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1720ed-409c-4840-b617-4b0406c113f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:20.438108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1720ed-409c-4840-b617-4b0406c113f2 admin admin] PATH_INFO: `/v3/projects/0ae763ebde3f4deca586965dc728f1bd/users/cce341aa9c9b4910a1ced37bae65561a/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:20.450636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16daa523-e65a-4429-b6ea-eedcbdd6e3b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:20.457879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16daa523-e65a-4429-b6ea-eedcbdd6e3b5 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:20.480702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 836/1719] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:20.481985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daa523-e65a-4429-b6ea-eedcbdd6e3b5 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:20.484567 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1720ed-409c-4840-b617-4b0406c113f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae763ebde3f4deca586965dc728f1bd, user_id=cce341aa9c9b4910a1ced37bae65561a, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:20.493577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1720ed-409c-4840-b617-4b0406c113f2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:20.513940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daa523-e65a-4429-b6ea-eedcbdd6e3b5 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:20.539024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bf318-4bc9-4bfc-9abc-ca559500c245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:20.540107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bf318-4bc9-4bfc-9abc-ca559500c245 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:20.540546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bf318-4bc9-4bfc-9abc-ca559500c245 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:20.541045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bf318-4bc9-4bfc-9abc-ca559500c245 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:20.542089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bf318-4bc9-4bfc-9abc-ca559500c245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:20.542571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bf318-4bc9-4bfc-9abc-ca559500c245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:20.544190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bf318-4bc9-4bfc-9abc-ca559500c245 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:20.560270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 884/1720] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:20 2022] PUT /identity/v3/projects/0ae763ebde3f4deca586965dc728f1bd/users/cce341aa9c9b4910a1ced37bae65561a/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:20.560825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1720ed-409c-4840-b617-4b0406c113f2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:20.582633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:20.592440 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 837/1721] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:20.597646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bf318-4bc9-4bfc-9abc-ca559500c245 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:20.608504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-684bf318-4bc9-4bfc-9abc-ca559500c245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:20.645219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bf318-4bc9-4bfc-9abc-ca559500c245 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:20.749041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1720ed-409c-4840-b617-4b0406c113f2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:20.776721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3507c74-7309-425b-88b5-610cc5c211ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:20.777511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3507c74-7309-425b-88b5-610cc5c211ac admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:20.777801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3507c74-7309-425b-88b5-610cc5c211ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:20.778078 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3507c74-7309-425b-88b5-610cc5c211ac admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:20.782313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3507c74-7309-425b-88b5-610cc5c211ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:20.784912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3507c74-7309-425b-88b5-610cc5c211ac admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:20.857177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:20.858192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:20.858476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:20.858750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] PATH_INFO: `/v3/users/c637cb6b8dfa4a308d90f35e9b7d0d92` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:20.860110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c637cb6b8dfa4a308d90f35e9b7d0d92)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:20.862896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:20.872459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 885/1722] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:20 2022] POST /identity/v3/projects => generated 371 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:20.877744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eca32e-f15a-4a9a-bc7b-8ed8c36ab7ba None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:20.878042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eca32e-f15a-4a9a-bc7b-8ed8c36ab7ba None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:20.878317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eca32e-f15a-4a9a-bc7b-8ed8c36ab7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:20.964101 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-23eca32e-f15a-4a9a-bc7b-8ed8c36ab7ba None None] MFA Rules not processed for user `cce341aa9c9b4910a1ced37bae65561a`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:20.970090 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-23eca32e-f15a-4a9a-bc7b-8ed8c36ab7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:21.007182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c637cb6b8dfa4a308d90f35e9b7d0d92'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:21.020455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c637cb6b8dfa4a308d90f35e9b7d0d92'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:21.047618 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c637cb6b8dfa4a308d90f35e9b7d0d92'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:21.118857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c637cb6b8dfa4a308d90f35e9b7d0d92'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:21.120185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c637cb6b8dfa4a308d90f35e9b7d0d92'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:21.149503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 838/1723] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:20 2022] DELETE /identity/v3/users/c637cb6b8dfa4a308d90f35e9b7d0d92 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:21.151139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:21.169960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:21.190671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:21.191019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 886/1724] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:21.192127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eca32e-f15a-4a9a-bc7b-8ed8c36ab7ba None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:21.402936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3732c8d-1e0c-4d88-a3ed-9ad4f200e65d admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:21.424825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7a6f2-31f1-4c80-807e-690ad786649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:21.425967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7a6f2-31f1-4c80-807e-690ad786649b admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:21.426511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7a6f2-31f1-4c80-807e-690ad786649b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:21.426797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7a6f2-31f1-4c80-807e-690ad786649b admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:21.427518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7a6f2-31f1-4c80-807e-690ad786649b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:21.429490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7a6f2-31f1-4c80-807e-690ad786649b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:21.499916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eca32e-f15a-4a9a-bc7b-8ed8c36ab7ba None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:21.515131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaa824-8bc5-480d-8d13-ba3c94e10ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:21.517004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaa824-8bc5-480d-8d13-ba3c94e10ccb admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:21.520183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaa824-8bc5-480d-8d13-ba3c94e10ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:21.520516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaa824-8bc5-480d-8d13-ba3c94e10ccb admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:21.521647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbaa824-8bc5-480d-8d13-ba3c94e10ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:21.524356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbaa824-8bc5-480d-8d13-ba3c94e10ccb admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:21.543950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 839/1725] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:21 2022] POST /identity/v3/projects => generated 381 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:21.551122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343cd1-83c2-4d04-a8e1-a74481d408ac None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:21.551416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343cd1-83c2-4d04-a8e1-a74481d408ac None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:21.551820 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343cd1-83c2-4d04-a8e1-a74481d408ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:21.585280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 887/1726] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:21 2022] POST /identity/v3/users => generated 286 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:21.588272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaa824-8bc5-480d-8d13-ba3c94e10ccb admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:21.593882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-dbbaa824-8bc5-480d-8d13-ba3c94e10ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:21.612964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-5a343cd1-83c2-4d04-a8e1-a74481d408ac None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:21.614332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5a343cd1-83c2-4d04-a8e1-a74481d408ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:21.658540 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaa824-8bc5-480d-8d13-ba3c94e10ccb admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:21.685696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:21.810741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48dd0b-7039-49f2-b297-6ddc5323c58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:21.811705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48dd0b-7039-49f2-b297-6ddc5323c58c admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:21.824048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48dd0b-7039-49f2-b297-6ddc5323c58c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:21.824460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48dd0b-7039-49f2-b297-6ddc5323c58c admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:21.825538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48dd0b-7039-49f2-b297-6ddc5323c58c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:21.828318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48dd0b-7039-49f2-b297-6ddc5323c58c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:21.851825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 840/1727] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:21.862975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a343cd1-83c2-4d04-a8e1-a74481d408ac None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:21.891149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 888/1728] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:21 2022] POST /identity/v3/projects => generated 363 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:21.893579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48dd0b-7039-49f2-b297-6ddc5323c58c admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:21.895984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ef48dd0b-7039-49f2-b297-6ddc5323c58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:21.900878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a343cd1-83c2-4d04-a8e1-a74481d408ac None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.035685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48dd0b-7039-49f2-b297-6ddc5323c58c admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.042863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587c6ea-7256-4cff-bf74-5dcd5b564b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:22.044068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c6ea-7256-4cff-bf74-5dcd5b564b82 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.044400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c6ea-7256-4cff-bf74-5dcd5b564b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.044680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587c6ea-7256-4cff-bf74-5dcd5b564b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:22.045467 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587c6ea-7256-4cff-bf74-5dcd5b564b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:22.047661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587c6ea-7256-4cff-bf74-5dcd5b564b82 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:22.072873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 889/1729] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:22.077932 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587c6ea-7256-4cff-bf74-5dcd5b564b82 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:22.109805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983dc07-238c-4007-8a52-4de4403ad5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:22.112433 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983dc07-238c-4007-8a52-4de4403ad5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.115252 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587c6ea-7256-4cff-bf74-5dcd5b564b82 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.115737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983dc07-238c-4007-8a52-4de4403ad5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.116650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983dc07-238c-4007-8a52-4de4403ad5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:22.120139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983dc07-238c-4007-8a52-4de4403ad5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:22.122349 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983dc07-238c-4007-8a52-4de4403ad5c0 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:22.136171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cce08d-28c7-4e7c-b02e-9beb09fab9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:22.136931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cce08d-28c7-4e7c-b02e-9beb09fab9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.146204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cce08d-28c7-4e7c-b02e-9beb09fab9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.146492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cce08d-28c7-4e7c-b02e-9beb09fab9c8 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:22.147380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cce08d-28c7-4e7c-b02e-9beb09fab9c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:22.151663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cce08d-28c7-4e7c-b02e-9beb09fab9c8 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:22.171380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 841/1730] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:21 2022] POST /identity/v3/users => generated 291 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:22.175334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983dc07-238c-4007-8a52-4de4403ad5c0 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:22.180510 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c983dc07-238c-4007-8a52-4de4403ad5c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:22.190229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 890/1731] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:22 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:22.191612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cce08d-28c7-4e7c-b02e-9beb09fab9c8 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:22.210038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cce08d-28c7-4e7c-b02e-9beb09fab9c8 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.217761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc04b62-b05a-40ec-beae-1e3265d7d1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:22.218742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc04b62-b05a-40ec-beae-1e3265d7d1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.219034 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc04b62-b05a-40ec-beae-1e3265d7d1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.219344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc04b62-b05a-40ec-beae-1e3265d7d1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:22.220746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc04b62-b05a-40ec-beae-1e3265d7d1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:22.223047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc04b62-b05a-40ec-beae-1e3265d7d1a7 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:22.252390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:22.266595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 891/1732] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:22.268285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc04b62-b05a-40ec-beae-1e3265d7d1a7 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:22.313454 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc04b62-b05a-40ec-beae-1e3265d7d1a7 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.328733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9061741-0222-40d4-b13e-9474e1fc9df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:22.337516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9061741-0222-40d4-b13e-9474e1fc9df2 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.338220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9061741-0222-40d4-b13e-9474e1fc9df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.339411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9061741-0222-40d4-b13e-9474e1fc9df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:22.341638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9061741-0222-40d4-b13e-9474e1fc9df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:22.351256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9061741-0222-40d4-b13e-9474e1fc9df2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:22.380898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983dc07-238c-4007-8a52-4de4403ad5c0 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.386219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31a7ce-6d44-484f-8f14-7434d74b5b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:22.387044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31a7ce-6d44-484f-8f14-7434d74b5b6f service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.387403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31a7ce-6d44-484f-8f14-7434d74b5b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.387825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31a7ce-6d44-484f-8f14-7434d74b5b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:22.403325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed31a7ce-6d44-484f-8f14-7434d74b5b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:22.405735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed31a7ce-6d44-484f-8f14-7434d74b5b6f service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:22.409807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 892/1733] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:22.413343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9061741-0222-40d4-b13e-9474e1fc9df2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:22.430546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 842/1734] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:22.433049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31a7ce-6d44-484f-8f14-7434d74b5b6f service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:22.442701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9061741-0222-40d4-b13e-9474e1fc9df2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.448920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd7135-d828-46f4-8418-cce61785377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:22.449918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd7135-d828-46f4-8418-cce61785377e admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.450380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd7135-d828-46f4-8418-cce61785377e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.450812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd7135-d828-46f4-8418-cce61785377e admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:22.452248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd7135-d828-46f4-8418-cce61785377e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:22.454195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd7135-d828-46f4-8418-cce61785377e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:22.468576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 893/1735] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:22.476015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd7135-d828-46f4-8418-cce61785377e admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:22.481913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31a7ce-6d44-484f-8f14-7434d74b5b6f service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.496365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd7135-d828-46f4-8418-cce61785377e admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.503141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab6bfc-b6f3-4978-a994-b93932e44a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:22.504794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6bfc-b6f3-4978-a994-b93932e44a62 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.505545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6bfc-b6f3-4978-a994-b93932e44a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.506042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6bfc-b6f3-4978-a994-b93932e44a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:22.507092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab6bfc-b6f3-4978-a994-b93932e44a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:22.509200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab6bfc-b6f3-4978-a994-b93932e44a62 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:22.509641 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a34185-447e-47d7-bb79-d8af1abd9ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:22.511630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a34185-447e-47d7-bb79-d8af1abd9ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.511900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a34185-447e-47d7-bb79-d8af1abd9ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.512183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a34185-447e-47d7-bb79-d8af1abd9ca6 admin admin] PATH_INFO: `/v3/projects/f8bddd5198f14347a30e35991b512181/users/0781df4c5b684ff39e4b7ed8cb725fa6/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:22.531995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 894/1736] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:22.534364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab6bfc-b6f3-4978-a994-b93932e44a62 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:22.548191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab6bfc-b6f3-4978-a994-b93932e44a62 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.555361 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155915e-b241-4095-aacd-ea86cad312e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:22.556512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155915e-b241-4095-aacd-ea86cad312e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.556927 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155915e-b241-4095-aacd-ea86cad312e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.557314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155915e-b241-4095-aacd-ea86cad312e7 admin admin] PATH_INFO: `/v3/projects/c601da03f2c44a3ebfcbad12d1014e51/users/e6fcb084af094907a274823e2eb79c41/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:22.588273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155915e-b241-4095-aacd-ea86cad312e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c601da03f2c44a3ebfcbad12d1014e51, user_id=e6fcb084af094907a274823e2eb79c41, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:22.591533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155915e-b241-4095-aacd-ea86cad312e7 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:22.597219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a34185-447e-47d7-bb79-d8af1abd9ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8bddd5198f14347a30e35991b512181, user_id=0781df4c5b684ff39e4b7ed8cb725fa6, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:22.605178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a34185-447e-47d7-bb79-d8af1abd9ca6 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:22.616590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 895/1737] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:22 2022] PUT /identity/v3/projects/c601da03f2c44a3ebfcbad12d1014e51/users/e6fcb084af094907a274823e2eb79c41/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:22.618046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155915e-b241-4095-aacd-ea86cad312e7 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:22.637654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:22.656420 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 843/1738] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:22 2022] PUT /identity/v3/projects/f8bddd5198f14347a30e35991b512181/users/0781df4c5b684ff39e4b7ed8cb725fa6/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:22.656420 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a34185-447e-47d7-bb79-d8af1abd9ca6 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:22.824451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155915e-b241-4095-aacd-ea86cad312e7 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.865063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a34185-447e-47d7-bb79-d8af1abd9ca6 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:22.882310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cab101-15df-454a-89c0-65f4f959c67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:22.883007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cab101-15df-454a-89c0-65f4f959c67c admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.883289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cab101-15df-454a-89c0-65f4f959c67c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.883570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cab101-15df-454a-89c0-65f4f959c67c admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:22.884314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cab101-15df-454a-89c0-65f4f959c67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:22.902554 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cab101-15df-454a-89c0-65f4f959c67c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:22.932330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 844/1739] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:22.933876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd71ea0-a9f8-4134-8598-d2a1cdad30cb None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:22.934261 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd71ea0-a9f8-4134-8598-d2a1cdad30cb None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:22.934809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd71ea0-a9f8-4134-8598-d2a1cdad30cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:23.032404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-7dd71ea0-a9f8-4134-8598-d2a1cdad30cb None None] MFA Rules not processed for user `0781df4c5b684ff39e4b7ed8cb725fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:23.034035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7dd71ea0-a9f8-4134-8598-d2a1cdad30cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:23.040939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f858787-0bd1-4de1-bc77-8181b641502e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:23.041232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f858787-0bd1-4de1-bc77-8181b641502e admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:23.041398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f858787-0bd1-4de1-bc77-8181b641502e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:23.041805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f858787-0bd1-4de1-bc77-8181b641502e admin admin] PATH_INFO: `/v3/projects/c1ff0382f07f4b7e9182e95237893905/users/7f96b1a233d942b0abb45e15ba318b9a/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:23.088248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f858787-0bd1-4de1-bc77-8181b641502e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ff0382f07f4b7e9182e95237893905, user_id=7f96b1a233d942b0abb45e15ba318b9a, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:23.090651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f858787-0bd1-4de1-bc77-8181b641502e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:23.127375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 896/1740] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:22 2022] PUT /identity/v3/projects/c1ff0382f07f4b7e9182e95237893905/users/7f96b1a233d942b0abb45e15ba318b9a/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:23.128203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f858787-0bd1-4de1-bc77-8181b641502e admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:23.201204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 845/1741] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:23.202583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00942a-f0ab-4a40-86d6-6d380a6b840b None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:23.202868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00942a-f0ab-4a40-86d6-6d380a6b840b None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:23.203141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00942a-f0ab-4a40-86d6-6d380a6b840b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:23.284030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-5b00942a-f0ab-4a40-86d6-6d380a6b840b None None] MFA Rules not processed for user `7f96b1a233d942b0abb45e15ba318b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:23.286047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5b00942a-f0ab-4a40-86d6-6d380a6b840b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:23.291962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f858787-0bd1-4de1-bc77-8181b641502e admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:23.308679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91589b4e-8b3c-44bb-856b-be3157146b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:23.312295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91589b4e-8b3c-44bb-856b-be3157146b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:23.313052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91589b4e-8b3c-44bb-856b-be3157146b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:23.313617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:23.317600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91589b4e-8b3c-44bb-856b-be3157146b4e admin admin] PATH_INFO: `/v3/projects/c601da03f2c44a3ebfcbad12d1014e51/users/e6fcb084af094907a274823e2eb79c41/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:23.330327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91589b4e-8b3c-44bb-856b-be3157146b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c601da03f2c44a3ebfcbad12d1014e51, user_id=e6fcb084af094907a274823e2eb79c41, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:23.355106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91589b4e-8b3c-44bb-856b-be3157146b4e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:23.438471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 897/1742] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:23 2022] PUT /identity/v3/projects/c601da03f2c44a3ebfcbad12d1014e51/users/e6fcb084af094907a274823e2eb79c41/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:23.441699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91589b4e-8b3c-44bb-856b-be3157146b4e admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:23.533150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:23.556510 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 846/1743] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:23.560957 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2492c79-f8b7-42e4-8701-bea7e7ff885e None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:23.562978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2492c79-f8b7-42e4-8701-bea7e7ff885e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:23.564359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2492c79-f8b7-42e4-8701-bea7e7ff885e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:23.648797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91589b4e-8b3c-44bb-856b-be3157146b4e admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:23.678040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbb455-b044-465b-904c-d3c4b393d559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:23.683102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb455-b044-465b-904c-d3c4b393d559 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:23.683102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb455-b044-465b-904c-d3c4b393d559 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:23.683102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb455-b044-465b-904c-d3c4b393d559 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:23.683102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbb455-b044-465b-904c-d3c4b393d559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:23.688306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c2492c79-f8b7-42e4-8701-bea7e7ff885e None None] MFA Rules not processed for user `e6fcb084af094907a274823e2eb79c41`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:23.690669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c2492c79-f8b7-42e4-8701-bea7e7ff885e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:23.698236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbb455-b044-465b-904c-d3c4b393d559 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:23.811401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 898/1744] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:23 2022] POST /identity/v3/projects => generated 371 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:23.814390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbb455-b044-465b-904c-d3c4b393d559 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:23.956572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbb455-b044-465b-904c-d3c4b393d559 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:23.961987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406164bf-f2c8-4f0e-a61f-919f9a956123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:23.964181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406164bf-f2c8-4f0e-a61f-919f9a956123 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:23.964719 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406164bf-f2c8-4f0e-a61f-919f9a956123 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:23.965272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406164bf-f2c8-4f0e-a61f-919f9a956123 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:23.966660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406164bf-f2c8-4f0e-a61f-919f9a956123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:23.969120 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406164bf-f2c8-4f0e-a61f-919f9a956123 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:24.008013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 847/1745] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:23 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:24.009387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2492c79-f8b7-42e4-8701-bea7e7ff885e None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:24.041780 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 899/1746] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:23 2022] POST /identity/v3/projects => generated 383 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:24.046320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406164bf-f2c8-4f0e-a61f-919f9a956123 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:24.049924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-406164bf-f2c8-4f0e-a61f-919f9a956123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:24.096886 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2492c79-f8b7-42e4-8701-bea7e7ff885e None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:24.148049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406164bf-f2c8-4f0e-a61f-919f9a956123 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:24.155888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e7db5-116f-4269-aa90-d244927c2038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:24.157031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e7db5-116f-4269-aa90-d244927c2038 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:24.157833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e7db5-116f-4269-aa90-d244927c2038 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:24.158330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e7db5-116f-4269-aa90-d244927c2038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:24.289825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:24.319553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e7db5-116f-4269-aa90-d244927c2038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:24.340856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e7db5-116f-4269-aa90-d244927c2038 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:24.440546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a3007-b395-457a-a737-556a2a1ae4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:24.441337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a3007-b395-457a-a737-556a2a1ae4fd admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:24.441570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a3007-b395-457a-a737-556a2a1ae4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:24.441855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a3007-b395-457a-a737-556a2a1ae4fd admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:24.442561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a3007-b395-457a-a737-556a2a1ae4fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:24.444515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a3007-b395-457a-a737-556a2a1ae4fd admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:24.452622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 900/1747] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:24 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:24.457019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e7db5-116f-4269-aa90-d244927c2038 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:24.505631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e7db5-116f-4269-aa90-d244927c2038 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:24.532087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 848/1748] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:24 2022] POST /identity/v3/users => generated 292 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:24.535708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a3007-b395-457a-a737-556a2a1ae4fd admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:24.538612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b9f62-d164-4015-a8d4-59770fb8bb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:24.541523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9f62-d164-4015-a8d4-59770fb8bb52 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:24.544980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9f62-d164-4015-a8d4-59770fb8bb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:24.546856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9f62-d164-4015-a8d4-59770fb8bb52 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:24.550515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b9f62-d164-4015-a8d4-59770fb8bb52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:24.559677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b9f62-d164-4015-a8d4-59770fb8bb52 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:24.580658 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a3007-b395-457a-a737-556a2a1ae4fd admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:24.591592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c6ec-59d3-4fd5-ac52-d0cd2ad6b421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:24.592693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c6ec-59d3-4fd5-ac52-d0cd2ad6b421 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:24.593643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c6ec-59d3-4fd5-ac52-d0cd2ad6b421 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:24.593939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c6ec-59d3-4fd5-ac52-d0cd2ad6b421 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:24.594818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322c6ec-59d3-4fd5-ac52-d0cd2ad6b421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:24.608651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322c6ec-59d3-4fd5-ac52-d0cd2ad6b421 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:24.637811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 849/1749] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:24.640131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c6ec-59d3-4fd5-ac52-d0cd2ad6b421 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:24.662368 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 901/1750] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:24 2022] POST /identity/v3/users => generated 298 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:24.667694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b9f62-d164-4015-a8d4-59770fb8bb52 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:24.679545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c6ec-59d3-4fd5-ac52-d0cd2ad6b421 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:24.732943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b9f62-d164-4015-a8d4-59770fb8bb52 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:24.747263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c19e8-ea50-45eb-876e-feae525e4c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:24.756201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c19e8-ea50-45eb-876e-feae525e4c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:24.756631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c19e8-ea50-45eb-876e-feae525e4c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:24.756921 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c19e8-ea50-45eb-876e-feae525e4c75 admin admin] PATH_INFO: `/v3/projects/9ece9ea9ef8b4cf89643439722e98f68/users/5f67100628504cf28c5a21de810c7b0d/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:24.771653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b05807-5cf0-4962-a7a6-c64e17ed9578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:24.774400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b05807-5cf0-4962-a7a6-c64e17ed9578 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:24.775703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b05807-5cf0-4962-a7a6-c64e17ed9578 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:24.777144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b05807-5cf0-4962-a7a6-c64e17ed9578 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:24.783955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b05807-5cf0-4962-a7a6-c64e17ed9578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:24.786051 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b05807-5cf0-4962-a7a6-c64e17ed9578 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:24.796604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:24.805946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c19e8-ea50-45eb-876e-feae525e4c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ece9ea9ef8b4cf89643439722e98f68, user_id=5f67100628504cf28c5a21de810c7b0d, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:24.812523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c19e8-ea50-45eb-876e-feae525e4c75 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:24.827897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 902/1751] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:24.830502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b05807-5cf0-4962-a7a6-c64e17ed9578 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:24.859313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 850/1752] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:24 2022] PUT /identity/v3/projects/9ece9ea9ef8b4cf89643439722e98f68/users/5f67100628504cf28c5a21de810c7b0d/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:24.860689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c19e8-ea50-45eb-876e-feae525e4c75 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:24.865637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b05807-5cf0-4962-a7a6-c64e17ed9578 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:25.025398 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c19e8-ea50-45eb-876e-feae525e4c75 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:25.043458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e71831-f93d-44a9-9afd-5a7a50aad68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:25.044349 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e71831-f93d-44a9-9afd-5a7a50aad68f admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:25.044644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e71831-f93d-44a9-9afd-5a7a50aad68f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:25.044930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e71831-f93d-44a9-9afd-5a7a50aad68f admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:25.045821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e71831-f93d-44a9-9afd-5a7a50aad68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:25.050124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e71831-f93d-44a9-9afd-5a7a50aad68f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:25.087272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 851/1753] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:25.088227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e71831-f93d-44a9-9afd-5a7a50aad68f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:25.119831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e71831-f93d-44a9-9afd-5a7a50aad68f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:25.138304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90d126-9597-492d-a56f-dff4c9a93192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:25.139239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90d126-9597-492d-a56f-dff4c9a93192 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:25.139523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90d126-9597-492d-a56f-dff4c9a93192 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:25.139802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90d126-9597-492d-a56f-dff4c9a93192 admin admin] PATH_INFO: `/v3/projects/9ece9ea9ef8b4cf89643439722e98f68/users/5f67100628504cf28c5a21de810c7b0d/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:25.148758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90d126-9597-492d-a56f-dff4c9a93192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ece9ea9ef8b4cf89643439722e98f68, user_id=5f67100628504cf28c5a21de810c7b0d, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:25.159534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90d126-9597-492d-a56f-dff4c9a93192 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:25.185013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466b91f-ffa7-4870-8e2d-b66924600d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:25.186066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466b91f-ffa7-4870-8e2d-b66924600d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:25.186357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466b91f-ffa7-4870-8e2d-b66924600d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:25.186638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466b91f-ffa7-4870-8e2d-b66924600d33 admin admin] PATH_INFO: `/v3/projects/64ce303d411c48198af8bb471f190f08/users/29976e5b8721456cb0c416b57530a012/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:25.231101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 852/1754] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:25 2022] PUT /identity/v3/projects/9ece9ea9ef8b4cf89643439722e98f68/users/5f67100628504cf28c5a21de810c7b0d/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:25.233638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3cd63-dd27-440f-ba8a-ea2262dfaf5e None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:25.233871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3cd63-dd27-440f-ba8a-ea2262dfaf5e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:25.234153 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3cd63-dd27-440f-ba8a-ea2262dfaf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:25.265519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466b91f-ffa7-4870-8e2d-b66924600d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ce303d411c48198af8bb471f190f08, user_id=29976e5b8721456cb0c416b57530a012, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:25.268270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466b91f-ffa7-4870-8e2d-b66924600d33 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:25.320751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-47c3cd63-dd27-440f-ba8a-ea2262dfaf5e None None] MFA Rules not processed for user `5f67100628504cf28c5a21de810c7b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:25.323151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-47c3cd63-dd27-440f-ba8a-ea2262dfaf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:25.346601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 903/1755] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:24 2022] PUT /identity/v3/projects/64ce303d411c48198af8bb471f190f08/users/29976e5b8721456cb0c416b57530a012/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:25.350571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466b91f-ffa7-4870-8e2d-b66924600d33 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:25.589766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 853/1756] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:25.600426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3cd63-dd27-440f-ba8a-ea2262dfaf5e None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:25.602785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-47c3cd63-dd27-440f-ba8a-ea2262dfaf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:25.648120 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466b91f-ffa7-4870-8e2d-b66924600d33 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:25.667377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6917a-e491-45f9-a89d-359d8eeea583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:25.668248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6917a-e491-45f9-a89d-359d8eeea583 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:25.668535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6917a-e491-45f9-a89d-359d8eeea583 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:25.668815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6917a-e491-45f9-a89d-359d8eeea583 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:25.674223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6917a-e491-45f9-a89d-359d8eeea583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:25.686385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6917a-e491-45f9-a89d-359d8eeea583 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:25.731139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 904/1757] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:25.734504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6917a-e491-45f9-a89d-359d8eeea583 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:25.771380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6917a-e491-45f9-a89d-359d8eeea583 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:25.775669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3cd63-dd27-440f-ba8a-ea2262dfaf5e None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:25.785308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cf7f8-8adb-44ca-b0ba-b7dd9ab2fa59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:25.788616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cf7f8-8adb-44ca-b0ba-b7dd9ab2fa59 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:25.788930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cf7f8-8adb-44ca-b0ba-b7dd9ab2fa59 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:25.790547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cf7f8-8adb-44ca-b0ba-b7dd9ab2fa59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:25.806076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3cdc0-75dd-4241-8160-0ab02589ebe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:25.807841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3cdc0-75dd-4241-8160-0ab02589ebe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:25.808568 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3cdc0-75dd-4241-8160-0ab02589ebe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:25.809119 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3cdc0-75dd-4241-8160-0ab02589ebe1 admin admin] PATH_INFO: `/v3/projects/64ce303d411c48198af8bb471f190f08/users/29976e5b8721456cb0c416b57530a012/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:25.815006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c3cdc0-75dd-4241-8160-0ab02589ebe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ce303d411c48198af8bb471f190f08, user_id=29976e5b8721456cb0c416b57530a012, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:25.821613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c3cdc0-75dd-4241-8160-0ab02589ebe1 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:25.826889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cf7f8-8adb-44ca-b0ba-b7dd9ab2fa59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:25.829377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cf7f8-8adb-44ca-b0ba-b7dd9ab2fa59 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:25.860857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 854/1758] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:49:25.903921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 905/1759] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:25 2022] PUT /identity/v3/projects/64ce303d411c48198af8bb471f190f08/users/29976e5b8721456cb0c416b57530a012/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:25.927053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aada6e-d2a0-4de7-8396-310619ff9f56 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:25.929229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aada6e-d2a0-4de7-8396-310619ff9f56 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:25.929837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aada6e-d2a0-4de7-8396-310619ff9f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:25.951754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:26.022571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-e5aada6e-d2a0-4de7-8396-310619ff9f56 None None] MFA Rules not processed for user `29976e5b8721456cb0c416b57530a012`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:26.024160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e5aada6e-d2a0-4de7-8396-310619ff9f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:26.048710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3cdc0-75dd-4241-8160-0ab02589ebe1 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:26.183933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:26.286162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 855/1760] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:26.297097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aada6e-d2a0-4de7-8396-310619ff9f56 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:26.326214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3cdc0-75dd-4241-8160-0ab02589ebe1 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:26.352924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:26.354063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:26.354493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:26.354911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] PATH_INFO: `/v3/users/cce341aa9c9b4910a1ced37bae65561a` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:26.356907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cce341aa9c9b4910a1ced37bae65561a)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:26.359047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:26.467705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aada6e-d2a0-4de7-8396-310619ff9f56 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:26.484219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42152ec0-14a8-4191-86b5-a503bf3dd412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:26.503602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42152ec0-14a8-4191-86b5-a503bf3dd412 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:26.503602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42152ec0-14a8-4191-86b5-a503bf3dd412 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:26.515483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42152ec0-14a8-4191-86b5-a503bf3dd412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:26.542996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42152ec0-14a8-4191-86b5-a503bf3dd412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:26.548654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42152ec0-14a8-4191-86b5-a503bf3dd412 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:26.557618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cce341aa9c9b4910a1ced37bae65561a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:26.585410 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 856/1761] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:26.599407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cce341aa9c9b4910a1ced37bae65561a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:26.641506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cce341aa9c9b4910a1ced37bae65561a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:26.641506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cce341aa9c9b4910a1ced37bae65561a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:26.665425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cce341aa9c9b4910a1ced37bae65561a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:26.694040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 906/1762] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:26 2022] DELETE /identity/v3/users/cce341aa9c9b4910a1ced37bae65561a => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:26.697252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a5bf-7c10-44ea-9dde-e563d471591c None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:26.698204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a5bf-7c10-44ea-9dde-e563d471591c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:26.698917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a5bf-7c10-44ea-9dde-e563d471591c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:26.746067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-41a7a5bf-7c10-44ea-9dde-e563d471591c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:26.747729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-41a7a5bf-7c10-44ea-9dde-e563d471591c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:26.909679 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 857/1763] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:26.945078 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:26.946782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:27.075488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca92a29-6ccd-4087-a292-e256d5409ba1 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:27.076023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca92a29-6ccd-4087-a292-e256d5409ba1 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:27.076591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca92a29-6ccd-4087-a292-e256d5409ba1 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:27.080401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 858/1764] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 20:49:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:49:27.082928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca92a29-6ccd-4087-a292-e256d5409ba1 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:27.119261 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4ca92a29-6ccd-4087-a292-e256d5409ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:27.212624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca92a29-6ccd-4087-a292-e256d5409ba1 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:27.225277 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3e9da-7022-45e7-9623-9821fea2c29f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:27.233103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cda36-056d-47d6-b383-b541fe712dea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:27.234208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8468dd-62d3-4805-8618-3db889ff5d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:27.235581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8468dd-62d3-4805-8618-3db889ff5d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:27.235870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cda36-056d-47d6-b383-b541fe712dea service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:27.236163 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cda36-056d-47d6-b383-b541fe712dea service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:27.236447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89cda36-056d-47d6-b383-b541fe712dea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:27.238404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8468dd-62d3-4805-8618-3db889ff5d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:27.240218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8468dd-62d3-4805-8618-3db889ff5d85 admin admin] PATH_INFO: `/v3/users/0781df4c5b684ff39e4b7ed8cb725fa6` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:27.243963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8468dd-62d3-4805-8618-3db889ff5d85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0781df4c5b684ff39e4b7ed8cb725fa6)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:27.246023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c89cda36-056d-47d6-b383-b541fe712dea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:27.254481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8468dd-62d3-4805-8618-3db889ff5d85 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:27.340313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:27.428233 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-3f8468dd-62d3-4805-8618-3db889ff5d85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0781df4c5b684ff39e4b7ed8cb725fa6'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:27.458309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-3f8468dd-62d3-4805-8618-3db889ff5d85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0781df4c5b684ff39e4b7ed8cb725fa6'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:27.466175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cda36-056d-47d6-b383-b541fe712dea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:27.468308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89cda36-056d-47d6-b383-b541fe712dea service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:27.482757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-3f8468dd-62d3-4805-8618-3db889ff5d85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0781df4c5b684ff39e4b7ed8cb725fa6'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:27.514813 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 907/1765] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:27.515531 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cda36-056d-47d6-b383-b541fe712dea service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:27.524232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c89cda36-056d-47d6-b383-b541fe712dea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:27.533096 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:27.544422 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-3f8468dd-62d3-4805-8618-3db889ff5d85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0781df4c5b684ff39e4b7ed8cb725fa6'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:27.545234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-3f8468dd-62d3-4805-8618-3db889ff5d85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0781df4c5b684ff39e4b7ed8cb725fa6'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:27.563708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 859/1766] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:27 2022] DELETE /identity/v3/users/0781df4c5b684ff39e4b7ed8cb725fa6 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:27.565752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb96fd-bd9e-4cae-8658-3d078e1c0754 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:27.566139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb96fd-bd9e-4cae-8658-3d078e1c0754 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:27.571200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb96fd-bd9e-4cae-8658-3d078e1c0754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:27.642767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-defb96fd-bd9e-4cae-8658-3d078e1c0754 None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:27.644343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-defb96fd-bd9e-4cae-8658-3d078e1c0754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:27.671969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89cda36-056d-47d6-b383-b541fe712dea service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:27.869005 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 860/1767] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 20:49:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:27.870561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2a083-b294-4071-ad06-9b3ca7413cc8 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:27.871482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2a083-b294-4071-ad06-9b3ca7413cc8 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:27.872714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2a083-b294-4071-ad06-9b3ca7413cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:27.879887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef156ce-31f5-4788-8db8-dabf3c963194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:27.881273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef156ce-31f5-4788-8db8-dabf3c963194 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:27.884670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef156ce-31f5-4788-8db8-dabf3c963194 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:27.884978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef156ce-31f5-4788-8db8-dabf3c963194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:27.902329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2ef156ce-31f5-4788-8db8-dabf3c963194 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:27.938783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-7ed2a083-b294-4071-ad06-9b3ca7413cc8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:27.941943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7ed2a083-b294-4071-ad06-9b3ca7413cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:28.148177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 861/1768] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:28.153147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2a083-b294-4071-ad06-9b3ca7413cc8 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:28.156965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7ed2a083-b294-4071-ad06-9b3ca7413cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:28.192725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef156ce-31f5-4788-8db8-dabf3c963194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:28.195931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef156ce-31f5-4788-8db8-dabf3c963194 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:28.216673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2a083-b294-4071-ad06-9b3ca7413cc8 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:28.249457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 908/1769] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:28.249978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a47bd-686f-47c3-b83e-8668b2d94146 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:28.249978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a47bd-686f-47c3-b83e-8668b2d94146 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:28.249978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a47bd-686f-47c3-b83e-8668b2d94146 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:28.249978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a47bd-686f-47c3-b83e-8668b2d94146 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:28.264432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef156ce-31f5-4788-8db8-dabf3c963194 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:28.266059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2ef156ce-31f5-4788-8db8-dabf3c963194 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:28.287515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a47bd-686f-47c3-b83e-8668b2d94146 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:28.290407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef156ce-31f5-4788-8db8-dabf3c963194 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:28.291817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a47bd-686f-47c3-b83e-8668b2d94146 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:28.304499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e43df3-4396-46be-9384-b5f285396364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:28.305187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e43df3-4396-46be-9384-b5f285396364 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:28.305501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e43df3-4396-46be-9384-b5f285396364 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:28.305781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e43df3-4396-46be-9384-b5f285396364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:28.351837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 862/1770] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:28.356136 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e43df3-4396-46be-9384-b5f285396364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:28.359130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e43df3-4396-46be-9384-b5f285396364 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:28.396182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 909/1771] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:49:29.478591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a47bd-686f-47c3-b83e-8668b2d94146 service nova] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:29.480137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bf0a47bd-686f-47c3-b83e-8668b2d94146 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:29.535943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a47bd-686f-47c3-b83e-8668b2d94146 service nova] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:29.561867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:29.581825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:29.582121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:29.582430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] PATH_INFO: `/v3/projects/800423f34bdc4dbd81a9874217d3ceaf` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:29.588026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=800423f34bdc4dbd81a9874217d3ceaf)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:29.591073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:29.680386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:29.752481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '800423f34bdc4dbd81a9874217d3ceaf', 'request_id': 'req-14464bc1-bfb1-4378-92c0-ee4048540244'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:29.770261 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '800423f34bdc4dbd81a9874217d3ceaf', 'request_id': 'req-14464bc1-bfb1-4378-92c0-ee4048540244'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:29.796227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '800423f34bdc4dbd81a9874217d3ceaf', 'request_id': 'req-14464bc1-bfb1-4378-92c0-ee4048540244'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:29.823308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] The token cache is being invalidate because project 800423f34bdc4dbd81a9874217d3ceaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:29.823601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:29.829941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-14464bc1-bfb1-4378-92c0-ee4048540244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:29.835176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 863/1772] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:29 2022] DELETE /identity/v3/projects/800423f34bdc4dbd81a9874217d3ceaf => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:29.840210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e43df3-4396-46be-9384-b5f285396364 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:29.852749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e43df3-4396-46be-9384-b5f285396364 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:30.033777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:30.066238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:30.067326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:30.067876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:30.068261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] PATH_INFO: `/v3/users/2a597573c6214add96f2222515ad15a2` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:30.072985 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a597573c6214add96f2222515ad15a2)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:30.075143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:30.253601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a597573c6214add96f2222515ad15a2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:30.284121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a597573c6214add96f2222515ad15a2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:30.313498 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a597573c6214add96f2222515ad15a2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:30.314473 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a597573c6214add96f2222515ad15a2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:30.328327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a597573c6214add96f2222515ad15a2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:30.342048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 910/1773] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:29 2022] DELETE /identity/v3/users/2a597573c6214add96f2222515ad15a2 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:30.762392 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ab0ea-5298-4f3e-82c3-882cca7920b2 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:30.762972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ab0ea-5298-4f3e-82c3-882cca7920b2 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:30.762972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ab0ea-5298-4f3e-82c3-882cca7920b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:30.806962 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-df8ab0ea-5298-4f3e-82c3-882cca7920b2 None None] MFA Rules not processed for user `af989ff6dc024ff19023e6f633915ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:30.808208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-df8ab0ea-5298-4f3e-82c3-882cca7920b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:30.904847 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 864/1774] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:31.235047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:31.237348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:31.443093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbf0fd-d394-44ee-8196-48e923307740 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:31.444788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ab0ea-5298-4f3e-82c3-882cca7920b2 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:31.446875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-df8ab0ea-5298-4f3e-82c3-882cca7920b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:31.468295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:31.471379 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:31.471906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:31.478309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ab0ea-5298-4f3e-82c3-882cca7920b2 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:31.479382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] PATH_INFO: `/v3/projects/0ae763ebde3f4deca586965dc728f1bd` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:31.481012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae763ebde3f4deca586965dc728f1bd)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:31.482989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:31.489922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbdc56-2f67-42e0-a3cd-92aa0135630e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:31.491052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbdc56-2f67-42e0-a3cd-92aa0135630e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:31.491494 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbdc56-2f67-42e0-a3cd-92aa0135630e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:31.491933 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbdc56-2f67-42e0-a3cd-92aa0135630e admin admin] PATH_INFO: `/v3/users/7f96b1a233d942b0abb45e15ba318b9a` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:31.497619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbdc56-2f67-42e0-a3cd-92aa0135630e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f96b1a233d942b0abb45e15ba318b9a)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:31.500020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbdc56-2f67-42e0-a3cd-92aa0135630e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:31.598442 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2ecbdc56-2f67-42e0-a3cd-92aa0135630e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f96b1a233d942b0abb45e15ba318b9a'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:31.612993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae763ebde3f4deca586965dc728f1bd', 'request_id': 'req-65b78169-61c7-4a65-a5bb-7dde197590fe'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:31.618536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2ecbdc56-2f67-42e0-a3cd-92aa0135630e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f96b1a233d942b0abb45e15ba318b9a'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:31.629649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae763ebde3f4deca586965dc728f1bd', 'request_id': 'req-65b78169-61c7-4a65-a5bb-7dde197590fe'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:31.630879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2ecbdc56-2f67-42e0-a3cd-92aa0135630e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f96b1a233d942b0abb45e15ba318b9a'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:31.654291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae763ebde3f4deca586965dc728f1bd', 'request_id': 'req-65b78169-61c7-4a65-a5bb-7dde197590fe'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:31.666743 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2ecbdc56-2f67-42e0-a3cd-92aa0135630e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f96b1a233d942b0abb45e15ba318b9a'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:31.672346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2ecbdc56-2f67-42e0-a3cd-92aa0135630e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f96b1a233d942b0abb45e15ba318b9a'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:31.674513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] The token cache is being invalidate because project 0ae763ebde3f4deca586965dc728f1bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:31.674893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:31.677297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:31.680761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 911/1775] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:31 2022] DELETE /identity/v3/projects/0ae763ebde3f4deca586965dc728f1bd => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:31.684168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 865/1776] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:31 2022] DELETE /identity/v3/users/7f96b1a233d942b0abb45e15ba318b9a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:31.685039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:31.688314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:31.688914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508b990-a98e-4714-9c69-1e65cbc8b766 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:31.689387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508b990-a98e-4714-9c69-1e65cbc8b766 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:31.689829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508b990-a98e-4714-9c69-1e65cbc8b766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:31.696647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:31.725324 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-6508b990-a98e-4714-9c69-1e65cbc8b766 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:31.726482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6508b990-a98e-4714-9c69-1e65cbc8b766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:31.754187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:31.849143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 866/1777] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:31.850732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7a2f9-a53d-469e-8ef6-8cc354a36905 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:31.851028 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7a2f9-a53d-469e-8ef6-8cc354a36905 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:31.851417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7a2f9-a53d-469e-8ef6-8cc354a36905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:31.877745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b78169-61c7-4a65-a5bb-7dde197590fe admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:31.885493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:31.886675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:31.887119 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:31.887553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] PATH_INFO: `/v3/users/e6fcb084af094907a274823e2eb79c41` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:31.888550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-7df7a2f9-a53d-469e-8ef6-8cc354a36905 None None] MFA Rules not processed for user `31790576f8734f75adbbd8e9c0d61721`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:31.889176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6fcb084af094907a274823e2eb79c41)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:31.890389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7df7a2f9-a53d-469e-8ef6-8cc354a36905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:31.891303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:31.966990 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6fcb084af094907a274823e2eb79c41'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:31.968934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 867/1778] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:31.970777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7a2f9-a53d-469e-8ef6-8cc354a36905 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:31.977486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7df7a2f9-a53d-469e-8ef6-8cc354a36905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:32.009346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7a2f9-a53d-469e-8ef6-8cc354a36905 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:32.021924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6fcb084af094907a274823e2eb79c41'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.034449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:32.035693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6fcb084af094907a274823e2eb79c41'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.036509 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6fcb084af094907a274823e2eb79c41'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.037184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:32.037715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:32.038220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] PATH_INFO: `/v3/projects/f8bddd5198f14347a30e35991b512181` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:32.039873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8bddd5198f14347a30e35991b512181)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:32.044749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:32.059215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6fcb084af094907a274823e2eb79c41'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.074820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 912/1779] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:31 2022] DELETE /identity/v3/users/e6fcb084af094907a274823e2eb79c41 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:32.074820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:32.074820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:32.157727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8bddd5198f14347a30e35991b512181', 'request_id': 'req-d0c1603c-13d1-4753-950a-6730f2b72759'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.175760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8bddd5198f14347a30e35991b512181', 'request_id': 'req-d0c1603c-13d1-4753-950a-6730f2b72759'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.183691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8bddd5198f14347a30e35991b512181', 'request_id': 'req-d0c1603c-13d1-4753-950a-6730f2b72759'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.203749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e804a-7780-449f-9414-82d2ed8b663d admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:32.207369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] The token cache is being invalidate because project f8bddd5198f14347a30e35991b512181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:32.207662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.208828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.213416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 868/1780] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:31 2022] DELETE /identity/v3/projects/f8bddd5198f14347a30e35991b512181 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:32.217873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:32.220890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:32.315742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:32.331495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:32.373513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1603c-13d1-4753-950a-6730f2b72759 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:32.379619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:32.380714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:32.380834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:32.381099 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] PATH_INFO: `/v3/users/5f67100628504cf28c5a21de810c7b0d` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:32.383056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f67100628504cf28c5a21de810c7b0d)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:32.385277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:32.441771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0efa68-b55d-412d-885b-b219deeb7674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:32.446281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0efa68-b55d-412d-885b-b219deeb7674 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:32.447886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0efa68-b55d-412d-885b-b219deeb7674 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:32.448254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0efa68-b55d-412d-885b-b219deeb7674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:32.457074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ad0efa68-b55d-412d-885b-b219deeb7674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:32.488948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0efa68-b55d-412d-885b-b219deeb7674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:32.497156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f67100628504cf28c5a21de810c7b0d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.517524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f67100628504cf28c5a21de810c7b0d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.518219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0efa68-b55d-412d-885b-b219deeb7674 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:32.526905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f67100628504cf28c5a21de810c7b0d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.542339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f67100628504cf28c5a21de810c7b0d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.543188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f67100628504cf28c5a21de810c7b0d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.555523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 869/1781] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:32 2022] DELETE /identity/v3/users/5f67100628504cf28c5a21de810c7b0d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:32.556810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:32.558327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:32.576280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 913/1782] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:32.577966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed0047-a6f0-457e-89e4-02a8efc2700c None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:32.580410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed0047-a6f0-457e-89e4-02a8efc2700c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:32.581782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed0047-a6f0-457e-89e4-02a8efc2700c None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:32.586669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 914/1783] 10.222.0.70 () {54 vars in 912 bytes} [Tue Jul 19 20:49:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:49:32.591841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b14ef-3752-40a4-a536-b623d6d6b707 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:32.592794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b14ef-3752-40a4-a536-b623d6d6b707 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:32.593340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b14ef-3752-40a4-a536-b623d6d6b707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:32.638885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-336b14ef-3752-40a4-a536-b623d6d6b707 None None] MFA Rules not processed for user `fb186ee2e3c94ca7ad203c9c0e32e498`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:32.642943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-336b14ef-3752-40a4-a536-b623d6d6b707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:32.707178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319c305-47b0-433f-8a83-f6142c55c0f5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:32.714475 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:32.715811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:32.716337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:32.719263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] PATH_INFO: `/v3/projects/8e9749c8ff454aa69d9e11962973e95d` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:32.722151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e9749c8ff454aa69d9e11962973e95d)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:32.724420 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:32.769287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 915/1784] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:49:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:32.790554 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e9749c8ff454aa69d9e11962973e95d', 'request_id': 'req-a96b7013-2243-49c4-bd6a-b5914c362ea7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.797402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e9749c8ff454aa69d9e11962973e95d', 'request_id': 'req-a96b7013-2243-49c4-bd6a-b5914c362ea7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.806091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e9749c8ff454aa69d9e11962973e95d', 'request_id': 'req-a96b7013-2243-49c4-bd6a-b5914c362ea7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.808628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b14ef-3752-40a4-a536-b623d6d6b707 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:32.811292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-336b14ef-3752-40a4-a536-b623d6d6b707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:32.826066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b14ef-3752-40a4-a536-b623d6d6b707 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:32.829311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:32.841088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] The token cache is being invalidate because project 8e9749c8ff454aa69d9e11962973e95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:32.841620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.843591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a96b7013-2243-49c4-bd6a-b5914c362ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:32.846747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 870/1785] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:32 2022] DELETE /identity/v3/projects/8e9749c8ff454aa69d9e11962973e95d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:32.855599 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aa526-00bb-4d30-a3d7-bfd7aa24b34d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:32.857160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aa526-00bb-4d30-a3d7-bfd7aa24b34d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:32.857702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aa526-00bb-4d30-a3d7-bfd7aa24b34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:32.894193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-886aa526-00bb-4d30-a3d7-bfd7aa24b34d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:32.895645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-886aa526-00bb-4d30-a3d7-bfd7aa24b34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:32.994031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f588a-c4e4-4d0b-b973-21178561a3b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:32.994702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f588a-c4e4-4d0b-b973-21178561a3b7 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:32.994985 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f588a-c4e4-4d0b-b973-21178561a3b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:32.995260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f588a-c4e4-4d0b-b973-21178561a3b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:32.997265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-629f588a-c4e4-4d0b-b973-21178561a3b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:33.008377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f588a-c4e4-4d0b-b973-21178561a3b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.010020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f588a-c4e4-4d0b-b973-21178561a3b7 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:33.012147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 871/1786] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:33.020205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052c7a9-af65-489d-87cc-ac0d8492d0df None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.020631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052c7a9-af65-489d-87cc-ac0d8492d0df None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.021000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052c7a9-af65-489d-87cc-ac0d8492d0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.021339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 916/1787] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:33.043538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-4052c7a9-af65-489d-87cc-ac0d8492d0df None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:33.046226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4052c7a9-af65-489d-87cc-ac0d8492d0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:33.063054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 872/1788] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:33.063185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f588a-c4e4-4d0b-b973-21178561a3b7 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:33.064887 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052c7a9-af65-489d-87cc-ac0d8492d0df None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:33.066389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4052c7a9-af65-489d-87cc-ac0d8492d0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:33.079165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052c7a9-af65-489d-87cc-ac0d8492d0df None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:33.084661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:33.085592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.085861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.086136 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] PATH_INFO: `/v3/projects/c601da03f2c44a3ebfcbad12d1014e51` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.087505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c601da03f2c44a3ebfcbad12d1014e51)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.089256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f588a-c4e4-4d0b-b973-21178561a3b7 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:33.089486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:33.101831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9294c1a-1993-425b-8372-d62a6417669f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:33.103042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9294c1a-1993-425b-8372-d62a6417669f admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.103595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9294c1a-1993-425b-8372-d62a6417669f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.104389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9294c1a-1993-425b-8372-d62a6417669f admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.106274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9294c1a-1993-425b-8372-d62a6417669f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:33.106951 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9294c1a-1993-425b-8372-d62a6417669f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.108680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9294c1a-1993-425b-8372-d62a6417669f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:33.134283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 917/1789] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:33.135731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9294c1a-1993-425b-8372-d62a6417669f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:33.140290 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c9294c1a-1993-425b-8372-d62a6417669f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:33.157142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c601da03f2c44a3ebfcbad12d1014e51', 'request_id': 'req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:33.165451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c601da03f2c44a3ebfcbad12d1014e51', 'request_id': 'req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:33.169031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9294c1a-1993-425b-8372-d62a6417669f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:33.173614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c601da03f2c44a3ebfcbad12d1014e51', 'request_id': 'req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:33.192327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] The token cache is being invalidate because project c601da03f2c44a3ebfcbad12d1014e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:33.192626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:33.193743 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:33.208478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 873/1790] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:33 2022] DELETE /identity/v3/projects/c601da03f2c44a3ebfcbad12d1014e51 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:33.209564 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:33.211592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:33.251859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:33.252748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.253030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.253300 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] PATH_INFO: `/v3/projects/9ece9ea9ef8b4cf89643439722e98f68` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.254855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ece9ea9ef8b4cf89643439722e98f68)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.257136 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:33.289218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:33.313148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd62e7-3e7d-4d7b-87d6-4609c121acb0 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:33.325931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ece9ea9ef8b4cf89643439722e98f68', 'request_id': 'req-99e09dc6-e625-433c-b959-a9d183249654'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:33.348600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ece9ea9ef8b4cf89643439722e98f68', 'request_id': 'req-99e09dc6-e625-433c-b959-a9d183249654'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:33.353088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:33.358748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ece9ea9ef8b4cf89643439722e98f68', 'request_id': 'req-99e09dc6-e625-433c-b959-a9d183249654'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:33.388141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] The token cache is being invalidate because project 9ece9ea9ef8b4cf89643439722e98f68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:33.388443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:33.389187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-99e09dc6-e625-433c-b959-a9d183249654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:33.396740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 918/1791] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:33 2022] DELETE /identity/v3/projects/9ece9ea9ef8b4cf89643439722e98f68 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:33.396740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de459edc-a3a8-4bd5-a219-06341d819567 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.396740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de459edc-a3a8-4bd5-a219-06341d819567 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.396740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de459edc-a3a8-4bd5-a219-06341d819567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.421270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-de459edc-a3a8-4bd5-a219-06341d819567 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:33.423565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-de459edc-a3a8-4bd5-a219-06341d819567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:33.452557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995c89d-8c46-4613-967a-55b07a3ac12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:33.454354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995c89d-8c46-4613-967a-55b07a3ac12f admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.455817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995c89d-8c46-4613-967a-55b07a3ac12f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.456179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995c89d-8c46-4613-967a-55b07a3ac12f admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.456979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995c89d-8c46-4613-967a-55b07a3ac12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.459529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995c89d-8c46-4613-967a-55b07a3ac12f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:33.515486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 874/1792] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:33 2022] POST /identity/v3/projects => generated 369 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:33.517010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2508b-9f6d-430e-ae29-65c9f1d5ee41 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.517299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2508b-9f6d-430e-ae29-65c9f1d5ee41 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.519354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2508b-9f6d-430e-ae29-65c9f1d5ee41 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.520893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 875/1793] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 20:49:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:49:33.521918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc0587-5b77-426f-825c-f8a63adfaf93 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.522202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc0587-5b77-426f-825c-f8a63adfaf93 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.522479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc0587-5b77-426f-825c-f8a63adfaf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.554630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-fcfc0587-5b77-426f-825c-f8a63adfaf93 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:33.555914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-fcfc0587-5b77-426f-825c-f8a63adfaf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:33.577430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 919/1794] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:33.577430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de459edc-a3a8-4bd5-a219-06341d819567 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:33.577430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-de459edc-a3a8-4bd5-a219-06341d819567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:33.612660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de459edc-a3a8-4bd5-a219-06341d819567 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:33.685587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ad8d9-736b-4cc1-ac99-e0a67e25756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:33.686475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ad8d9-736b-4cc1-ac99-e0a67e25756a admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.686836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ad8d9-736b-4cc1-ac99-e0a67e25756a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.687184 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ad8d9-736b-4cc1-ac99-e0a67e25756a admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.687888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ad8d9-736b-4cc1-ac99-e0a67e25756a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.690624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ad8d9-736b-4cc1-ac99-e0a67e25756a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:33.713504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 876/1795] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:33.713504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7670-b8bc-4cde-9aa2-3a843922c1d8 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.713504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7670-b8bc-4cde-9aa2-3a843922c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.713504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7670-b8bc-4cde-9aa2-3a843922c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.720224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 920/1796] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:33 2022] POST /identity/v3/users => generated 285 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:33.721697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1f288-6093-4cc1-bedd-5bac7d321574 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.722043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1f288-6093-4cc1-bedd-5bac7d321574 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.722429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1f288-6093-4cc1-bedd-5bac7d321574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.736978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-352e7670-b8bc-4cde-9aa2-3a843922c1d8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:33.739032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-352e7670-b8bc-4cde-9aa2-3a843922c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:33.748450 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-96f1f288-6093-4cc1-bedd-5bac7d321574 None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:33.749704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-96f1f288-6093-4cc1-bedd-5bac7d321574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:33.750153 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 877/1797] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:33.752233 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80ef31-cbf9-4c18-9cb6-8107c3f67475 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.752733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80ef31-cbf9-4c18-9cb6-8107c3f67475 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.753220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80ef31-cbf9-4c18-9cb6-8107c3f67475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.781936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-be80ef31-cbf9-4c18-9cb6-8107c3f67475 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:33.783658 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-be80ef31-cbf9-4c18-9cb6-8107c3f67475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:33.802167 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 878/1798] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:33.804255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80ef31-cbf9-4c18-9cb6-8107c3f67475 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:33.811309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:33.822324 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80ef31-cbf9-4c18-9cb6-8107c3f67475 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:33.830602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea93640-9c4d-4a73-80e7-860bbdb51ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:33.832811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea93640-9c4d-4a73-80e7-860bbdb51ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.833325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea93640-9c4d-4a73-80e7-860bbdb51ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.833970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea93640-9c4d-4a73-80e7-860bbdb51ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.835255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea93640-9c4d-4a73-80e7-860bbdb51ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.837768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea93640-9c4d-4a73-80e7-860bbdb51ea5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:33.859926 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 879/1799] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:33.861685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea93640-9c4d-4a73-80e7-860bbdb51ea5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:33.876741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 921/1800] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 20:49:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:33.878574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1f288-6093-4cc1-bedd-5bac7d321574 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:33.887745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea93640-9c4d-4a73-80e7-860bbdb51ea5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:33.889623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1f288-6093-4cc1-bedd-5bac7d321574 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:33.895332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4b043-5e94-4e81-9c3f-1fe2d04c37b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:33.896021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b043-5e94-4e81-9c3f-1fe2d04c37b2 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.896429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b043-5e94-4e81-9c3f-1fe2d04c37b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.896712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b043-5e94-4e81-9c3f-1fe2d04c37b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.897462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243ac69-5eec-48d3-bc0e-f01bd99a9bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:33.897704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4b043-5e94-4e81-9c3f-1fe2d04c37b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:33.897851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4b043-5e94-4e81-9c3f-1fe2d04c37b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.898555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243ac69-5eec-48d3-bc0e-f01bd99a9bfd admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.899079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243ac69-5eec-48d3-bc0e-f01bd99a9bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.899453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243ac69-5eec-48d3-bc0e-f01bd99a9bfd admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.900071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4b043-5e94-4e81-9c3f-1fe2d04c37b2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:33.900552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243ac69-5eec-48d3-bc0e-f01bd99a9bfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:33.901004 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243ac69-5eec-48d3-bc0e-f01bd99a9bfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.903960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243ac69-5eec-48d3-bc0e-f01bd99a9bfd admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:33.915994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 922/1801] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:33.917819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4b043-5e94-4e81-9c3f-1fe2d04c37b2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:33.939673 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4b043-5e94-4e81-9c3f-1fe2d04c37b2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:33.945993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb322dc-68d9-4100-a9f7-99817e96c28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:33.947188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb322dc-68d9-4100-a9f7-99817e96c28a admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.947667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb322dc-68d9-4100-a9f7-99817e96c28a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.948141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb322dc-68d9-4100-a9f7-99817e96c28a admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.949189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb322dc-68d9-4100-a9f7-99817e96c28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.951941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb322dc-68d9-4100-a9f7-99817e96c28a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:33.953226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 880/1802] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:33.955781 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243ac69-5eec-48d3-bc0e-f01bd99a9bfd admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:33.965644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 923/1803] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:33.966919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb322dc-68d9-4100-a9f7-99817e96c28a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:33.974249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243ac69-5eec-48d3-bc0e-f01bd99a9bfd admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:33.980626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb322dc-68d9-4100-a9f7-99817e96c28a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:33.982592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd770fd-79d2-4b0f-808a-7b7509843bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:33.983546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd770fd-79d2-4b0f-808a-7b7509843bdd admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.983994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd770fd-79d2-4b0f-808a-7b7509843bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.984234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd770fd-79d2-4b0f-808a-7b7509843bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.984950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd770fd-79d2-4b0f-808a-7b7509843bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.989497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f973610-b5d3-4861-8251-931bb5155953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:33.990446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973610-b5d3-4861-8251-931bb5155953 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:33.990877 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973610-b5d3-4861-8251-931bb5155953 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:33.991675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973610-b5d3-4861-8251-931bb5155953 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:33.992133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd770fd-79d2-4b0f-808a-7b7509843bdd admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:33.992763 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f973610-b5d3-4861-8251-931bb5155953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:33.994857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f973610-b5d3-4861-8251-931bb5155953 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:34.031185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 924/1804] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:33 2022] POST /identity/v3/projects => generated 363 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:34.033205 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f973610-b5d3-4861-8251-931bb5155953 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:34.046100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 881/1805] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:33 2022] POST /identity/v3/projects => generated 357 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:34.047276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd770fd-79d2-4b0f-808a-7b7509843bdd admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:34.122791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f973610-b5d3-4861-8251-931bb5155953 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:34.200566 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:34.229657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd770fd-79d2-4b0f-808a-7b7509843bdd admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:34.239312 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73753fd4-c300-46f2-bec7-6045a5d7d7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:34.240842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73753fd4-c300-46f2-bec7-6045a5d7d7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:34.241307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73753fd4-c300-46f2-bec7-6045a5d7d7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:34.241773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73753fd4-c300-46f2-bec7-6045a5d7d7c5 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:34.242847 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73753fd4-c300-46f2-bec7-6045a5d7d7c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:34.245990 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73753fd4-c300-46f2-bec7-6045a5d7d7c5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:34.276016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669d00a-d462-41bb-93c3-b2a7797aa76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:34.278943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d00a-d462-41bb-93c3-b2a7797aa76b admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:34.279752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d00a-d462-41bb-93c3-b2a7797aa76b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:34.280294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d00a-d462-41bb-93c3-b2a7797aa76b admin admin] PATH_INFO: `/v3/projects/abe585184af940f7a997e3a212da46ba/users/efcc6b848f094ea7ba1dfcc656904da5/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:34.292898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 882/1806] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:34 2022] POST /identity/v3/users => generated 282 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:34.294313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73753fd4-c300-46f2-bec7-6045a5d7d7c5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:34.310398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669d00a-d462-41bb-93c3-b2a7797aa76b admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe585184af940f7a997e3a212da46ba, user_id=efcc6b848f094ea7ba1dfcc656904da5, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:34.313634 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669d00a-d462-41bb-93c3-b2a7797aa76b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:34.323035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73753fd4-c300-46f2-bec7-6045a5d7d7c5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:34.341608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1297c90-9708-420d-b82d-ed4dd0d1d902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:34.343190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1297c90-9708-420d-b82d-ed4dd0d1d902 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:34.343706 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1297c90-9708-420d-b82d-ed4dd0d1d902 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:34.350908 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1297c90-9708-420d-b82d-ed4dd0d1d902 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:34.352508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1297c90-9708-420d-b82d-ed4dd0d1d902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:34.353117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 925/1807] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:34 2022] PUT /identity/v3/projects/abe585184af940f7a997e3a212da46ba/users/efcc6b848f094ea7ba1dfcc656904da5/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:34.354383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669d00a-d462-41bb-93c3-b2a7797aa76b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:34.357149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1297c90-9708-420d-b82d-ed4dd0d1d902 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:34.363176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e669d00a-d462-41bb-93c3-b2a7797aa76b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:34.417414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 883/1808] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:34 2022] POST /identity/v3/users => generated 279 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:34.418691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1297c90-9708-420d-b82d-ed4dd0d1d902 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:34.420150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:34.434250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1297c90-9708-420d-b82d-ed4dd0d1d902 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:34.532004 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ab82b-2ff4-4493-a998-d15222a285a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:34.533463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ab82b-2ff4-4493-a998-d15222a285a4 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:34.533848 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ab82b-2ff4-4493-a998-d15222a285a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:34.534292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ab82b-2ff4-4493-a998-d15222a285a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:34.535073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ab82b-2ff4-4493-a998-d15222a285a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:34.537015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ab82b-2ff4-4493-a998-d15222a285a4 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:34.552078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 884/1809] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:34.553540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529cbd2-90c7-4eb1-ba47-cc4f795ac883 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:34.553843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529cbd2-90c7-4eb1-ba47-cc4f795ac883 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:34.554124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529cbd2-90c7-4eb1-ba47-cc4f795ac883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:34.587758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c529cbd2-90c7-4eb1-ba47-cc4f795ac883 None None] MFA Rules not processed for user `efcc6b848f094ea7ba1dfcc656904da5`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:34.589055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c529cbd2-90c7-4eb1-ba47-cc4f795ac883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:34.589464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669d00a-d462-41bb-93c3-b2a7797aa76b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:34.602274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:34.603537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:34.604097 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:34.604524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] PATH_INFO: `/v3/projects/c1ff0382f07f4b7e9182e95237893905` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:34.607023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ff0382f07f4b7e9182e95237893905)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:34.617055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:34.670883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 885/1810] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:34.671913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529cbd2-90c7-4eb1-ba47-cc4f795ac883 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:34.684190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529cbd2-90c7-4eb1-ba47-cc4f795ac883 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:34.694950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce4270-2063-4dfa-bdd2-a615eba70d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:34.695914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4270-2063-4dfa-bdd2-a615eba70d81 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:34.697484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4270-2063-4dfa-bdd2-a615eba70d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:34.698110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4270-2063-4dfa-bdd2-a615eba70d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:34.702417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce4270-2063-4dfa-bdd2-a615eba70d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:34.709417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce4270-2063-4dfa-bdd2-a615eba70d81 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:34.740238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 886/1811] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:34.755104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ff0382f07f4b7e9182e95237893905', 'request_id': 'req-f906fcb9-d233-45d0-a79e-6ff34c98282f'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:34.757738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce4270-2063-4dfa-bdd2-a615eba70d81 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:34.762322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-cfce4270-2063-4dfa-bdd2-a615eba70d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:34.769287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ff0382f07f4b7e9182e95237893905', 'request_id': 'req-f906fcb9-d233-45d0-a79e-6ff34c98282f'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:34.778618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ff0382f07f4b7e9182e95237893905', 'request_id': 'req-f906fcb9-d233-45d0-a79e-6ff34c98282f'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:34.795521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] The token cache is being invalidate because project c1ff0382f07f4b7e9182e95237893905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:34.795820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:34.796530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:34.798843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 926/1812] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:34 2022] DELETE /identity/v3/projects/c1ff0382f07f4b7e9182e95237893905 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:34.799726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:34.801426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:34.839332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:34.899903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906fcb9-d233-45d0-a79e-6ff34c98282f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:34.900754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce4270-2063-4dfa-bdd2-a615eba70d81 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:34.906601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826f624-6bda-49fc-be7a-19232ecee48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:34.907336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826f624-6bda-49fc-be7a-19232ecee48f admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:34.907678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826f624-6bda-49fc-be7a-19232ecee48f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:34.907959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826f624-6bda-49fc-be7a-19232ecee48f admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:34.908965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826f624-6bda-49fc-be7a-19232ecee48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:34.911237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826f624-6bda-49fc-be7a-19232ecee48f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:34.928548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 927/1813] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:34.928548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826f624-6bda-49fc-be7a-19232ecee48f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:34.929227 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5826f624-6bda-49fc-be7a-19232ecee48f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:34.946499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826f624-6bda-49fc-be7a-19232ecee48f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:34.952294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e40bb-fa1d-4ee3-ac04-928093b595ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:34.953171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e40bb-fa1d-4ee3-ac04-928093b595ec admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:34.953497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e40bb-fa1d-4ee3-ac04-928093b595ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:34.953773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e40bb-fa1d-4ee3-ac04-928093b595ec admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:34.956171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e40bb-fa1d-4ee3-ac04-928093b595ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:34.958527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e40bb-fa1d-4ee3-ac04-928093b595ec admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:35.007501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 928/1814] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:34 2022] POST /identity/v3/projects => generated 367 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:35.009138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e40bb-fa1d-4ee3-ac04-928093b595ec admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:35.012349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-dd7e40bb-fa1d-4ee3-ac04-928093b595ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:35.030719 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35469f90-884b-4926-8e97-fed3fa4020cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:35.031738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35469f90-884b-4926-8e97-fed3fa4020cb service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:35.032098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35469f90-884b-4926-8e97-fed3fa4020cb service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:35.032486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35469f90-884b-4926-8e97-fed3fa4020cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:35.035841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-35469f90-884b-4926-8e97-fed3fa4020cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:35.045574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:35.073696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35469f90-884b-4926-8e97-fed3fa4020cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:35.075736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35469f90-884b-4926-8e97-fed3fa4020cb service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:35.108602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 887/1815] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:35.111373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35469f90-884b-4926-8e97-fed3fa4020cb service nova] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:35.119454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-35469f90-884b-4926-8e97-fed3fa4020cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:35.156346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35469f90-884b-4926-8e97-fed3fa4020cb service nova] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:35.162653 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:35.163880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:35.164237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:35.164593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] PATH_INFO: `/v3/users/29976e5b8721456cb0c416b57530a012` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:35.166525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29976e5b8721456cb0c416b57530a012)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:35.167641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e40bb-fa1d-4ee3-ac04-928093b595ec admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:35.172480 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:35.181860 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a5c5e-2b6a-4204-b0df-f7def2676abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:35.183183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a5c5e-2b6a-4204-b0df-f7def2676abe admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:35.185556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a5c5e-2b6a-4204-b0df-f7def2676abe admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:35.186083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a5c5e-2b6a-4204-b0df-f7def2676abe admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:35.188671 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a5c5e-2b6a-4204-b0df-f7def2676abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:35.197213 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a5c5e-2b6a-4204-b0df-f7def2676abe admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:35.237311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 929/1816] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:35.237311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a5c5e-2b6a-4204-b0df-f7def2676abe admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:35.266278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a5c5e-2b6a-4204-b0df-f7def2676abe admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:35.274880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bf570-c5ba-4aa6-b9be-035c005c3fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:35.275788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bf570-c5ba-4aa6-b9be-035c005c3fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:35.276070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bf570-c5ba-4aa6-b9be-035c005c3fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:35.276343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bf570-c5ba-4aa6-b9be-035c005c3fc6 admin admin] PATH_INFO: `/v3/projects/dea56860adb34241bed55cef10d337dd/users/d61e7f6848cd4efc919b1b360a1600d1/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:35.282891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29976e5b8721456cb0c416b57530a012'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:35.293812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29976e5b8721456cb0c416b57530a012'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:35.302879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bf570-c5ba-4aa6-b9be-035c005c3fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea56860adb34241bed55cef10d337dd, user_id=d61e7f6848cd4efc919b1b360a1600d1, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:35.306475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8bf570-c5ba-4aa6-b9be-035c005c3fc6 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:35.313199 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29976e5b8721456cb0c416b57530a012'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:35.328561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 930/1817] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:35 2022] PUT /identity/v3/projects/dea56860adb34241bed55cef10d337dd/users/d61e7f6848cd4efc919b1b360a1600d1/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:35.329591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bf570-c5ba-4aa6-b9be-035c005c3fc6 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:35.339277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29976e5b8721456cb0c416b57530a012'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:35.340687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29976e5b8721456cb0c416b57530a012'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:35.351057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 888/1818] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:35 2022] DELETE /identity/v3/users/29976e5b8721456cb0c416b57530a012 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:35.352196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:35.354320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:35.427100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:35.458565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8bf570-c5ba-4aa6-b9be-035c005c3fc6 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:35.496420 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4110d-e832-47f4-880c-a45bbdff95d0 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:35.502237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb91937-0bd3-4274-83d6-d8b015d12bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:35.503757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb91937-0bd3-4274-83d6-d8b015d12bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:35.504122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb91937-0bd3-4274-83d6-d8b015d12bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:35.504608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb91937-0bd3-4274-83d6-d8b015d12bc4 admin admin] PATH_INFO: `/v3/projects/40ef448d42fb4eb786f24409f00d808c/users/f6a4b7a1b7ad4e2db44cd55f5485aca9/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:35.529686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb91937-0bd3-4274-83d6-d8b015d12bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ef448d42fb4eb786f24409f00d808c, user_id=f6a4b7a1b7ad4e2db44cd55f5485aca9, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:35.533702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb91937-0bd3-4274-83d6-d8b015d12bc4 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:35.612180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 889/1819] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:35 2022] PUT /identity/v3/projects/40ef448d42fb4eb786f24409f00d808c/users/f6a4b7a1b7ad4e2db44cd55f5485aca9/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:35.615862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303db2ea-1314-46c5-a0fb-a178b3e41cd8 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:35.617713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303db2ea-1314-46c5-a0fb-a178b3e41cd8 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:35.618023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303db2ea-1314-46c5-a0fb-a178b3e41cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:35.639696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3189067-4b60-4102-b951-086e77a0dfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:35.640696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189067-4b60-4102-b951-086e77a0dfaf admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:35.640994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189067-4b60-4102-b951-086e77a0dfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:35.641281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189067-4b60-4102-b951-086e77a0dfaf admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:35.642290 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3189067-4b60-4102-b951-086e77a0dfaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:35.645117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3189067-4b60-4102-b951-086e77a0dfaf admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:35.670901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 931/1820] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:35 2022] POST /identity/v3/users => generated 290 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:35.672369 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f543d9-93f9-461f-adbe-064dcce9469d None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:35.672667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f543d9-93f9-461f-adbe-064dcce9469d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:35.672999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f543d9-93f9-461f-adbe-064dcce9469d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:35.697988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-303db2ea-1314-46c5-a0fb-a178b3e41cd8 None None] MFA Rules not processed for user `d61e7f6848cd4efc919b1b360a1600d1`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:35.699362 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-303db2ea-1314-46c5-a0fb-a178b3e41cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:35.713079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a1f543d9-93f9-461f-adbe-064dcce9469d None None] MFA Rules not processed for user `f6a4b7a1b7ad4e2db44cd55f5485aca9`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:35.714456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a1f543d9-93f9-461f-adbe-064dcce9469d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:35.750023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:35.789010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 890/1821] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:35 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:35.790063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303db2ea-1314-46c5-a0fb-a178b3e41cd8 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:35.791809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-303db2ea-1314-46c5-a0fb-a178b3e41cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:35.803989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 932/1822] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:35 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:35.808707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f543d9-93f9-461f-adbe-064dcce9469d None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:35.810414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a1f543d9-93f9-461f-adbe-064dcce9469d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:35.892343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303db2ea-1314-46c5-a0fb-a178b3e41cd8 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:35.897479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0087f59-f827-4dcd-b95e-fc0716535e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:35.898195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0087f59-f827-4dcd-b95e-fc0716535e6b admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:35.898538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0087f59-f827-4dcd-b95e-fc0716535e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:35.898894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0087f59-f827-4dcd-b95e-fc0716535e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:35.899707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0087f59-f827-4dcd-b95e-fc0716535e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:35.901424 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0087f59-f827-4dcd-b95e-fc0716535e6b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:35.913003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f543d9-93f9-461f-adbe-064dcce9469d None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:35.915160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 891/1823] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:35.920878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69969bc2-ab91-41a2-a1dc-903f4c3b03f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:35.921696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69969bc2-ab91-41a2-a1dc-903f4c3b03f2 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:35.922051 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69969bc2-ab91-41a2-a1dc-903f4c3b03f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:35.922417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69969bc2-ab91-41a2-a1dc-903f4c3b03f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:35.923218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69969bc2-ab91-41a2-a1dc-903f4c3b03f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:35.925413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69969bc2-ab91-41a2-a1dc-903f4c3b03f2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:35.930866 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0087f59-f827-4dcd-b95e-fc0716535e6b admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:35.949559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0087f59-f827-4dcd-b95e-fc0716535e6b admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:35.957130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada7e96-fd82-4cca-b249-2f44d93cc831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:35.958012 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7e96-fd82-4cca-b249-2f44d93cc831 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:35.958301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7e96-fd82-4cca-b249-2f44d93cc831 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:35.958672 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada7e96-fd82-4cca-b249-2f44d93cc831 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:35.959515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada7e96-fd82-4cca-b249-2f44d93cc831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:35.962036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada7e96-fd82-4cca-b249-2f44d93cc831 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:35.964270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 933/1824] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:35 2022] POST /identity/v3/projects => generated 363 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:35.965002 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69969bc2-ab91-41a2-a1dc-903f4c3b03f2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.000218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 892/1825] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:35 2022] POST /identity/v3/projects => generated 355 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:36.000874 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada7e96-fd82-4cca-b249-2f44d93cc831 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.029660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:36.050866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69969bc2-ab91-41a2-a1dc-903f4c3b03f2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.093835 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada7e96-fd82-4cca-b249-2f44d93cc831 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.099451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f42ec2-529b-40c1-bb48-2003c7542f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.100237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f42ec2-529b-40c1-bb48-2003c7542f1f admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.100417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f42ec2-529b-40c1-bb48-2003c7542f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.100695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f42ec2-529b-40c1-bb48-2003c7542f1f admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.101425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f42ec2-529b-40c1-bb48-2003c7542f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.103885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f42ec2-529b-40c1-bb48-2003c7542f1f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.111802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:36.125956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8985d-18fc-4643-a065-4118509ae8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.127336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8985d-18fc-4643-a065-4118509ae8c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.127620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8985d-18fc-4643-a065-4118509ae8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.127908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8985d-18fc-4643-a065-4118509ae8c0 admin admin] PATH_INFO: `/v3/projects/ce1087799293447484f34833f6ce60ee/users/19f9d56cea8a4211b2fe3bbcccfeff7e/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.129643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 893/1826] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:36 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:36.130585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f42ec2-529b-40c1-bb48-2003c7542f1f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.143041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f42ec2-529b-40c1-bb48-2003c7542f1f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.148826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f919965-8af2-477b-8f98-2a41483a4936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.149671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f919965-8af2-477b-8f98-2a41483a4936 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.149805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f919965-8af2-477b-8f98-2a41483a4936 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.150032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f919965-8af2-477b-8f98-2a41483a4936 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.151215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f919965-8af2-477b-8f98-2a41483a4936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.153349 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f919965-8af2-477b-8f98-2a41483a4936 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.154983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8985d-18fc-4643-a065-4118509ae8c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1087799293447484f34833f6ce60ee, user_id=19f9d56cea8a4211b2fe3bbcccfeff7e, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.157901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8985d-18fc-4643-a065-4118509ae8c0 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.181066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 934/1827] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:35 2022] PUT /identity/v3/projects/ce1087799293447484f34833f6ce60ee/users/19f9d56cea8a4211b2fe3bbcccfeff7e/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:36.181066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8985d-18fc-4643-a065-4118509ae8c0 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.199697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 894/1828] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:36 2022] POST /identity/v3/users => generated 284 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:36.199697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f919965-8af2-477b-8f98-2a41483a4936 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.220764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f919965-8af2-477b-8f98-2a41483a4936 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.288683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8985d-18fc-4643-a065-4118509ae8c0 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.293622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba809b-4485-4cc9-a3c6-ad91ceae4b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.294304 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba809b-4485-4cc9-a3c6-ad91ceae4b46 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.294593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba809b-4485-4cc9-a3c6-ad91ceae4b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.294873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba809b-4485-4cc9-a3c6-ad91ceae4b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.295624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba809b-4485-4cc9-a3c6-ad91ceae4b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.297292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba809b-4485-4cc9-a3c6-ad91ceae4b46 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.308111 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 935/1829] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:36.311561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba809b-4485-4cc9-a3c6-ad91ceae4b46 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.316449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158408d-3024-42f6-a4df-4b0689f26632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.317412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158408d-3024-42f6-a4df-4b0689f26632 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.317797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158408d-3024-42f6-a4df-4b0689f26632 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.318039 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158408d-3024-42f6-a4df-4b0689f26632 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.318895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158408d-3024-42f6-a4df-4b0689f26632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.320680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158408d-3024-42f6-a4df-4b0689f26632 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.322267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba809b-4485-4cc9-a3c6-ad91ceae4b46 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.328374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be317e-de0a-4ecc-b014-d953f8a12863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.329025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be317e-de0a-4ecc-b014-d953f8a12863 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.329988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be317e-de0a-4ecc-b014-d953f8a12863 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.330256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be317e-de0a-4ecc-b014-d953f8a12863 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.331397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be317e-de0a-4ecc-b014-d953f8a12863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.333417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be317e-de0a-4ecc-b014-d953f8a12863 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.337434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 895/1830] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:36.338238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158408d-3024-42f6-a4df-4b0689f26632 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.340971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8158408d-3024-42f6-a4df-4b0689f26632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:36.349983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 936/1831] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:36.352499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be317e-de0a-4ecc-b014-d953f8a12863 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.355287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158408d-3024-42f6-a4df-4b0689f26632 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.360155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.361111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.361429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.361715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] PATH_INFO: `/v3/projects/64ce303d411c48198af8bb471f190f08` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.363044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] RBAC: Authorizing `identity:delete_project(project_id=64ce303d411c48198af8bb471f190f08)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.364854 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be317e-de0a-4ecc-b014-d953f8a12863 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.365109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.386173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17bfcf-6419-4c86-a331-bd44a25b6a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.397891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17bfcf-6419-4c86-a331-bd44a25b6a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.398347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17bfcf-6419-4c86-a331-bd44a25b6a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.399027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17bfcf-6419-4c86-a331-bd44a25b6a54 admin admin] PATH_INFO: `/v3/projects/0a97c90093e9456aa75132cbc3c85902/users/52f1df6a3a6c43729ccd561453a55b15/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.417466 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17bfcf-6419-4c86-a331-bd44a25b6a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a97c90093e9456aa75132cbc3c85902, user_id=52f1df6a3a6c43729ccd561453a55b15, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.420267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17bfcf-6419-4c86-a331-bd44a25b6a54 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.430045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64ce303d411c48198af8bb471f190f08', 'request_id': 'req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:36.440596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64ce303d411c48198af8bb471f190f08', 'request_id': 'req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:36.445431 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 937/1832] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:36 2022] PUT /identity/v3/projects/0a97c90093e9456aa75132cbc3c85902/users/52f1df6a3a6c43729ccd561453a55b15/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:36.447848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17bfcf-6419-4c86-a331-bd44a25b6a54 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.453425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64ce303d411c48198af8bb471f190f08', 'request_id': 'req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:36.548679 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] The token cache is being invalidate because project 64ce303d411c48198af8bb471f190f08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:36.549856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:36.551454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:36.557907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 896/1833] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:36 2022] DELETE /identity/v3/projects/64ce303d411c48198af8bb471f190f08 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:36.557907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.558256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17bfcf-6419-4c86-a331-bd44a25b6a54 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.560168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:36.563424 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538d335-90d6-4df2-9240-b81857d526f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.564888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538d335-90d6-4df2-9240-b81857d526f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.565166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538d335-90d6-4df2-9240-b81857d526f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.565469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538d335-90d6-4df2-9240-b81857d526f9 admin admin] PATH_INFO: `/v3/projects/ce1087799293447484f34833f6ce60ee/users/19f9d56cea8a4211b2fe3bbcccfeff7e/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.572381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538d335-90d6-4df2-9240-b81857d526f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1087799293447484f34833f6ce60ee, user_id=19f9d56cea8a4211b2fe3bbcccfeff7e, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.577444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538d335-90d6-4df2-9240-b81857d526f9 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.583631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:36.602014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 938/1834] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:36 2022] PUT /identity/v3/projects/ce1087799293447484f34833f6ce60ee/users/19f9d56cea8a4211b2fe3bbcccfeff7e/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:36.603288 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538d335-90d6-4df2-9240-b81857d526f9 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.605334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0538d335-90d6-4df2-9240-b81857d526f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:36.650125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:36.665625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d0ee0-d6f6-4c1f-96b9-46645c1fe84c admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.694221 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538d335-90d6-4df2-9240-b81857d526f9 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.700141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480817f-c98d-42cf-a97d-45238c997cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.700822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480817f-c98d-42cf-a97d-45238c997cef admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.701109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480817f-c98d-42cf-a97d-45238c997cef admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.701412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480817f-c98d-42cf-a97d-45238c997cef admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.702158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480817f-c98d-42cf-a97d-45238c997cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.704787 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480817f-c98d-42cf-a97d-45238c997cef admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.721456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 939/1835] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:36.722722 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931fc79-40f0-4617-8147-854950e5d89f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.723005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931fc79-40f0-4617-8147-854950e5d89f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.723280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931fc79-40f0-4617-8147-854950e5d89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.734108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e685f-b3a7-4b55-9f80-9c4ef00bd537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.735019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e685f-b3a7-4b55-9f80-9c4ef00bd537 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.735304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e685f-b3a7-4b55-9f80-9c4ef00bd537 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.736108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e685f-b3a7-4b55-9f80-9c4ef00bd537 admin admin] PATH_INFO: `/v3/projects/c6ab56d8b453443ca35b0eb4c6a33b9f/users/95c8f2010d4a4e4897fa73e39b64a1ce/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.745455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-4931fc79-40f0-4617-8147-854950e5d89f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:36.746626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4931fc79-40f0-4617-8147-854950e5d89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:36.754671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e685f-b3a7-4b55-9f80-9c4ef00bd537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ab56d8b453443ca35b0eb4c6a33b9f, user_id=95c8f2010d4a4e4897fa73e39b64a1ce, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.755698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 940/1836] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:36.756996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234b6f7-dd49-4109-b030-9cfee371a678 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.757286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234b6f7-dd49-4109-b030-9cfee371a678 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.757605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234b6f7-dd49-4109-b030-9cfee371a678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.758776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e685f-b3a7-4b55-9f80-9c4ef00bd537 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.780037 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 897/1837] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:36 2022] PUT /identity/v3/projects/c6ab56d8b453443ca35b0eb4c6a33b9f/users/95c8f2010d4a4e4897fa73e39b64a1ce/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:36.782119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e685f-b3a7-4b55-9f80-9c4ef00bd537 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.808476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-c234b6f7-dd49-4109-b030-9cfee371a678 None None] MFA Rules not processed for user `19f9d56cea8a4211b2fe3bbcccfeff7e`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:36.810832 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c234b6f7-dd49-4109-b030-9cfee371a678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:36.895170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e685f-b3a7-4b55-9f80-9c4ef00bd537 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.901164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e0b71-401d-48ac-b7f8-299308d3d182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:36.902043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0b71-401d-48ac-b7f8-299308d3d182 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.902930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0b71-401d-48ac-b7f8-299308d3d182 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.902930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e0b71-401d-48ac-b7f8-299308d3d182 admin admin] PATH_INFO: `/v3/projects/0a97c90093e9456aa75132cbc3c85902/users/52f1df6a3a6c43729ccd561453a55b15/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.905108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e0b71-401d-48ac-b7f8-299308d3d182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a97c90093e9456aa75132cbc3c85902, user_id=52f1df6a3a6c43729ccd561453a55b15, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:36.909721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e0b71-401d-48ac-b7f8-299308d3d182 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:36.921930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 898/1838] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:36 2022] PUT /identity/v3/projects/0a97c90093e9456aa75132cbc3c85902/users/52f1df6a3a6c43729ccd561453a55b15/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:36.927388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a74dbd-0df6-4f7f-850f-70819daf6b63 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:36.927999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a74dbd-0df6-4f7f-850f-70819daf6b63 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:36.928512 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a74dbd-0df6-4f7f-850f-70819daf6b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:36.946278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 941/1839] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:36.947318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234b6f7-dd49-4109-b030-9cfee371a678 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:36.972641 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-44a74dbd-0df6-4f7f-850f-70819daf6b63 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:36.978837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234b6f7-dd49-4109-b030-9cfee371a678 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:36.989534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-44a74dbd-0df6-4f7f-850f-70819daf6b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:37.067867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:37.109243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c469ff2-ecac-4bfb-a782-46a5dc98cfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.111875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c469ff2-ecac-4bfb-a782-46a5dc98cfa4 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.112185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c469ff2-ecac-4bfb-a782-46a5dc98cfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.112727 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c469ff2-ecac-4bfb-a782-46a5dc98cfa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.113521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c469ff2-ecac-4bfb-a782-46a5dc98cfa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.117906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c469ff2-ecac-4bfb-a782-46a5dc98cfa4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:37.119511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 899/1840] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:37.121643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66ac51-b0a6-4961-9c5a-4f5a95b5fa44 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.121860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66ac51-b0a6-4961-9c5a-4f5a95b5fa44 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.122279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66ac51-b0a6-4961-9c5a-4f5a95b5fa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.144901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 942/1841] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:37.145692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c469ff2-ecac-4bfb-a782-46a5dc98cfa4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:37.147477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5c469ff2-ecac-4bfb-a782-46a5dc98cfa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:37.180148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-3e66ac51-b0a6-4961-9c5a-4f5a95b5fa44 None None] MFA Rules not processed for user `52f1df6a3a6c43729ccd561453a55b15`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:37.181618 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3e66ac51-b0a6-4961-9c5a-4f5a95b5fa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:37.222257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:37.241652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c469ff2-ecac-4bfb-a782-46a5dc98cfa4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:37.247343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318c1c8-c372-4c26-8ffa-7a7fed5cdcee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.247925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318c1c8-c372-4c26-8ffa-7a7fed5cdcee service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.248408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318c1c8-c372-4c26-8ffa-7a7fed5cdcee service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.248682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318c1c8-c372-4c26-8ffa-7a7fed5cdcee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.296272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 900/1842] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:37 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:37.302397 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66ac51-b0a6-4961-9c5a-4f5a95b5fa44 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:37.316774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66ac51-b0a6-4961-9c5a-4f5a95b5fa44 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:37.323384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fd7e3-771b-4e2e-ae6f-607841393df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.324198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fd7e3-771b-4e2e-ae6f-607841393df1 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.324559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fd7e3-771b-4e2e-ae6f-607841393df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.324908 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fd7e3-771b-4e2e-ae6f-607841393df1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.326076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fd7e3-771b-4e2e-ae6f-607841393df1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:37.326590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fd7e3-771b-4e2e-ae6f-607841393df1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.328371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fd7e3-771b-4e2e-ae6f-607841393df1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:37.341814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318c1c8-c372-4c26-8ffa-7a7fed5cdcee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.343972 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318c1c8-c372-4c26-8ffa-7a7fed5cdcee service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:37.352286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 901/1843] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:37.352286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fd7e3-771b-4e2e-ae6f-607841393df1 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:37.356915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 943/1844] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:37.357996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318c1c8-c372-4c26-8ffa-7a7fed5cdcee service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:37.359522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e318c1c8-c372-4c26-8ffa-7a7fed5cdcee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:37.373595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318c1c8-c372-4c26-8ffa-7a7fed5cdcee service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:37.378667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b93a1f-7e95-4d79-81fc-32214b7744e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.379335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b93a1f-7e95-4d79-81fc-32214b7744e9 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.379622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b93a1f-7e95-4d79-81fc-32214b7744e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.379953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b93a1f-7e95-4d79-81fc-32214b7744e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.387138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fd7e3-771b-4e2e-ae6f-607841393df1 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:37.391205 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b93a1f-7e95-4d79-81fc-32214b7744e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.392892 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b93a1f-7e95-4d79-81fc-32214b7744e9 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:37.403902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682786-4564-4e30-bb85-1d6b078f2dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.405424 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682786-4564-4e30-bb85-1d6b078f2dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.405916 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682786-4564-4e30-bb85-1d6b078f2dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.407411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682786-4564-4e30-bb85-1d6b078f2dbc admin admin] PATH_INFO: `/v3/projects/c6ab56d8b453443ca35b0eb4c6a33b9f/users/95c8f2010d4a4e4897fa73e39b64a1ce/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.407583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 944/1845] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:37.408429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b93a1f-7e95-4d79-81fc-32214b7744e9 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:37.413741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d682786-4564-4e30-bb85-1d6b078f2dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ab56d8b453443ca35b0eb4c6a33b9f, user_id=95c8f2010d4a4e4897fa73e39b64a1ce, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.420821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d682786-4564-4e30-bb85-1d6b078f2dbc admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:37.429421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b93a1f-7e95-4d79-81fc-32214b7744e9 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:37.443929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf13278-0649-4340-89f5-4acb1f55e4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.444624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf13278-0649-4340-89f5-4acb1f55e4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.444920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf13278-0649-4340-89f5-4acb1f55e4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.445191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf13278-0649-4340-89f5-4acb1f55e4e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.448432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 902/1846] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:37 2022] PUT /identity/v3/projects/c6ab56d8b453443ca35b0eb4c6a33b9f/users/95c8f2010d4a4e4897fa73e39b64a1ce/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:37.449555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf13278-0649-4340-89f5-4acb1f55e4e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.451638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf13278-0649-4340-89f5-4acb1f55e4e5 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:37.460548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8f107-e353-4059-a2cf-0495254dc1a0 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.460993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8f107-e353-4059-a2cf-0495254dc1a0 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.461170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8f107-e353-4059-a2cf-0495254dc1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.501779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-abe8f107-e353-4059-a2cf-0495254dc1a0 None None] MFA Rules not processed for user `95c8f2010d4a4e4897fa73e39b64a1ce`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:37.503297 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-abe8f107-e353-4059-a2cf-0495254dc1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:37.540653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 945/1847] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:37 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:37.542352 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf13278-0649-4340-89f5-4acb1f55e4e5 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:37.648270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf13278-0649-4340-89f5-4acb1f55e4e5 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:37.655482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111ddc3-306a-48e2-9b93-75dd07a7c7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.656688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111ddc3-306a-48e2-9b93-75dd07a7c7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.657233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111ddc3-306a-48e2-9b93-75dd07a7c7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.658362 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111ddc3-306a-48e2-9b93-75dd07a7c7f9 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.659065 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111ddc3-306a-48e2-9b93-75dd07a7c7f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.661459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111ddc3-306a-48e2-9b93-75dd07a7c7f9 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:37.663492 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 903/1848] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:37 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:37.664443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8f107-e353-4059-a2cf-0495254dc1a0 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:37.698725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 946/1849] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:37 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:37.699665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:37.699847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111ddc3-306a-48e2-9b93-75dd07a7c7f9 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:37.748153 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8f107-e353-4059-a2cf-0495254dc1a0 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:37.755728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:37.761579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111ddc3-306a-48e2-9b93-75dd07a7c7f9 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:37.766017 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531300a-2f9d-43b8-8af5-b58b8d76b33c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.766947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531300a-2f9d-43b8-8af5-b58b8d76b33c service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.767232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531300a-2f9d-43b8-8af5-b58b8d76b33c service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.767516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531300a-2f9d-43b8-8af5-b58b8d76b33c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.800988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0d7e1-80da-4a1c-a5aa-422d4b9e8e8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.802108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0d7e1-80da-4a1c-a5aa-422d4b9e8e8d service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.802388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0d7e1-80da-4a1c-a5aa-422d4b9e8e8d service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.802661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0d7e1-80da-4a1c-a5aa-422d4b9e8e8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.805203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c1b0d7e1-80da-4a1c-a5aa-422d4b9e8e8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:37.836753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531300a-2f9d-43b8-8af5-b58b8d76b33c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.838570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531300a-2f9d-43b8-8af5-b58b8d76b33c service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:37.850250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 947/1850] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:37.851172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531300a-2f9d-43b8-8af5-b58b8d76b33c service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:37.864077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531300a-2f9d-43b8-8af5-b58b8d76b33c service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:37.872686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76033a3-0054-4db9-ba79-01a9acceafb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.874005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76033a3-0054-4db9-ba79-01a9acceafb7 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.874455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76033a3-0054-4db9-ba79-01a9acceafb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.874720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76033a3-0054-4db9-ba79-01a9acceafb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.875462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76033a3-0054-4db9-ba79-01a9acceafb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.876372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0d7e1-80da-4a1c-a5aa-422d4b9e8e8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.877446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76033a3-0054-4db9-ba79-01a9acceafb7 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:37.878013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0d7e1-80da-4a1c-a5aa-422d4b9e8e8d service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:37.892744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 904/1851] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:37.892890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 948/1852] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:37.895001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0d7e1-80da-4a1c-a5aa-422d4b9e8e8d service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:37.906407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0d7e1-80da-4a1c-a5aa-422d4b9e8e8d service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:37.913323 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d596a5-6c22-432b-a068-a9aac81ea747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.914231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d596a5-6c22-432b-a068-a9aac81ea747 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.915000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d596a5-6c22-432b-a068-a9aac81ea747 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.915431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d596a5-6c22-432b-a068-a9aac81ea747 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.916333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d596a5-6c22-432b-a068-a9aac81ea747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.918964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d596a5-6c22-432b-a068-a9aac81ea747 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:37.939643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 905/1853] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:37.940129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76033a3-0054-4db9-ba79-01a9acceafb7 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:37.958164 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76033a3-0054-4db9-ba79-01a9acceafb7 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:37.964937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845789b-6322-47b3-990b-535de3955250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:37.965928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845789b-6322-47b3-990b-535de3955250 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:37.966214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845789b-6322-47b3-990b-535de3955250 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:37.966502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845789b-6322-47b3-990b-535de3955250 admin admin] PATH_INFO: `/v3/projects/eee0060807e441f58b892ff9d3845254/users/3216e261c913409b9b96fa1e782bd3df/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:37.990284 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845789b-6322-47b3-990b-535de3955250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee0060807e441f58b892ff9d3845254, user_id=3216e261c913409b9b96fa1e782bd3df, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:37.993146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845789b-6322-47b3-990b-535de3955250 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:38.011022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 949/1854] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:37 2022] PUT /identity/v3/projects/eee0060807e441f58b892ff9d3845254/users/3216e261c913409b9b96fa1e782bd3df/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:38.025707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a974a90-b74d-4a67-9a50-05edc6cf0007 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:38.026004 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a974a90-b74d-4a67-9a50-05edc6cf0007 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:38.026738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a974a90-b74d-4a67-9a50-05edc6cf0007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:38.042331 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845789b-6322-47b3-990b-535de3955250 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:38.062364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-4a974a90-b74d-4a67-9a50-05edc6cf0007 None None] MFA Rules not processed for user `3216e261c913409b9b96fa1e782bd3df`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:38.063958 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4a974a90-b74d-4a67-9a50-05edc6cf0007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:38.133824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845789b-6322-47b3-990b-535de3955250 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:38.138211 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53496901-c44c-4359-804e-14717e8a11d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:38.139037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53496901-c44c-4359-804e-14717e8a11d8 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:38.139423 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53496901-c44c-4359-804e-14717e8a11d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:38.139804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53496901-c44c-4359-804e-14717e8a11d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:38.144074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-53496901-c44c-4359-804e-14717e8a11d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:38.155252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 906/1855] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:38.160467 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a974a90-b74d-4a67-9a50-05edc6cf0007 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:38.171244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:38.229042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53496901-c44c-4359-804e-14717e8a11d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:38.231633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53496901-c44c-4359-804e-14717e8a11d8 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:38.241641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 950/1856] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:38 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:49:38.242210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a974a90-b74d-4a67-9a50-05edc6cf0007 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:38.247570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcd164-0079-4786-92d5-d448b5f367ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:38.248454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd164-0079-4786-92d5-d448b5f367ec admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:38.248658 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd164-0079-4786-92d5-d448b5f367ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:38.248904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd164-0079-4786-92d5-d448b5f367ec admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:38.249715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcd164-0079-4786-92d5-d448b5f367ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:38.253753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcd164-0079-4786-92d5-d448b5f367ec admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:38.285797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 907/1857] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:38 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:49:38.290509 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53496901-c44c-4359-804e-14717e8a11d8 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:38.304121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53496901-c44c-4359-804e-14717e8a11d8 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:38.319760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:38.384657 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9acc59-3510-4a36-a78f-0418804ded2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:38.385396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9acc59-3510-4a36-a78f-0418804ded2d admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:38.385655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9acc59-3510-4a36-a78f-0418804ded2d admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:38.387146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9acc59-3510-4a36-a78f-0418804ded2d admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:38.387146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9acc59-3510-4a36-a78f-0418804ded2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:38.388711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9acc59-3510-4a36-a78f-0418804ded2d admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:38.428527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 951/1858] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:38 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:38.432065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcd164-0079-4786-92d5-d448b5f367ec admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:38.444964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcd164-0079-4786-92d5-d448b5f367ec admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:38.450949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a4774-8ed6-470b-b2ea-37d5de24529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:38.451650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a4774-8ed6-470b-b2ea-37d5de24529e admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:38.451989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a4774-8ed6-470b-b2ea-37d5de24529e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:38.452221 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a4774-8ed6-470b-b2ea-37d5de24529e admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:38.453165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a4774-8ed6-470b-b2ea-37d5de24529e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:38.455354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a4774-8ed6-470b-b2ea-37d5de24529e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:38.478004 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 908/1859] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:38.484427 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9acc59-3510-4a36-a78f-0418804ded2d admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:38.493878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9acc59-3510-4a36-a78f-0418804ded2d admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:38.499124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067e25d-ad3c-4fc8-b0df-bdaeca8cadad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:38.500046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067e25d-ad3c-4fc8-b0df-bdaeca8cadad admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:38.500336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067e25d-ad3c-4fc8-b0df-bdaeca8cadad admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:38.500621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067e25d-ad3c-4fc8-b0df-bdaeca8cadad admin admin] PATH_INFO: `/v3/projects/8d5f50a377b14ca7a4daff8fe20d8ced/users/111d834e660a4ba68314fbeea8d37364/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:38.517119 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067e25d-ad3c-4fc8-b0df-bdaeca8cadad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5f50a377b14ca7a4daff8fe20d8ced, user_id=111d834e660a4ba68314fbeea8d37364, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:38.519943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067e25d-ad3c-4fc8-b0df-bdaeca8cadad admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:38.535952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 952/1860] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:38 2022] PUT /identity/v3/projects/8d5f50a377b14ca7a4daff8fe20d8ced/users/111d834e660a4ba68314fbeea8d37364/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:38.539292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a4774-8ed6-470b-b2ea-37d5de24529e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:38.552263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a4774-8ed6-470b-b2ea-37d5de24529e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:38.649528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad0340-bbd9-418f-8c8e-42594d8caed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:38.650259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0340-bbd9-418f-8c8e-42594d8caed5 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:38.650559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0340-bbd9-418f-8c8e-42594d8caed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:38.651139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad0340-bbd9-418f-8c8e-42594d8caed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:38.652394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad0340-bbd9-418f-8c8e-42594d8caed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:38.654075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad0340-bbd9-418f-8c8e-42594d8caed5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:38.667388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 909/1861] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:38.671349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067e25d-ad3c-4fc8-b0df-bdaeca8cadad admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:38.683512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067e25d-ad3c-4fc8-b0df-bdaeca8cadad admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:38.688624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b5032-616c-49ec-a37b-f60b95c1974a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:38.689803 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b5032-616c-49ec-a37b-f60b95c1974a admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:38.689803 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b5032-616c-49ec-a37b-f60b95c1974a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:38.690116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b5032-616c-49ec-a37b-f60b95c1974a admin admin] PATH_INFO: `/v3/projects/8d5f50a377b14ca7a4daff8fe20d8ced/users/111d834e660a4ba68314fbeea8d37364/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:38.694125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b5032-616c-49ec-a37b-f60b95c1974a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5f50a377b14ca7a4daff8fe20d8ced, user_id=111d834e660a4ba68314fbeea8d37364, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:38.696515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b5032-616c-49ec-a37b-f60b95c1974a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:38.709112 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 953/1862] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:38 2022] PUT /identity/v3/projects/8d5f50a377b14ca7a4daff8fe20d8ced/users/111d834e660a4ba68314fbeea8d37364/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:38.719794 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ed54e-baa4-43e2-9626-984c3f821762 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:38.720655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ed54e-baa4-43e2-9626-984c3f821762 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:38.720839 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ed54e-baa4-43e2-9626-984c3f821762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:38.758404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-248ed54e-baa4-43e2-9626-984c3f821762 None None] MFA Rules not processed for user `111d834e660a4ba68314fbeea8d37364`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:38.760390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-248ed54e-baa4-43e2-9626-984c3f821762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:38.892254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 910/1863] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:38.899004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b5032-616c-49ec-a37b-f60b95c1974a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:39.004927 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b5032-616c-49ec-a37b-f60b95c1974a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:39.008607 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a1770-b78f-40ab-892f-a6b28649bb04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:39.009274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1770-b78f-40ab-892f-a6b28649bb04 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:39.009591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1770-b78f-40ab-892f-a6b28649bb04 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:39.009867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1770-b78f-40ab-892f-a6b28649bb04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:39.025820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a1770-b78f-40ab-892f-a6b28649bb04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:39.028388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a1770-b78f-40ab-892f-a6b28649bb04 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:39.041738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 954/1864] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:39.225692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ed54e-baa4-43e2-9626-984c3f821762 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:39.231284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:39.320137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ed54e-baa4-43e2-9626-984c3f821762 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:39.346188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:39.351211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:39.352107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:39.356380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] PATH_INFO: `/v3/users/3216e261c913409b9b96fa1e782bd3df` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:39.359551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3216e261c913409b9b96fa1e782bd3df)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:39.370751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:39.490288 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3216e261c913409b9b96fa1e782bd3df'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:39.510830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3216e261c913409b9b96fa1e782bd3df'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:39.525614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3216e261c913409b9b96fa1e782bd3df'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:39.546305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3216e261c913409b9b96fa1e782bd3df'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:39.547648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3216e261c913409b9b96fa1e782bd3df'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:39.565565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 911/1865] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:39 2022] DELETE /identity/v3/users/3216e261c913409b9b96fa1e782bd3df => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:39.571927 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dac821-d01b-45fc-864d-fcf212ad72f3 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:39.572305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dac821-d01b-45fc-864d-fcf212ad72f3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:39.572582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dac821-d01b-45fc-864d-fcf212ad72f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:39.596964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-44dac821-d01b-45fc-864d-fcf212ad72f3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:39.597890 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-44dac821-d01b-45fc-864d-fcf212ad72f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:39.757967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 955/1866] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:39.768681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:39.770274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:39.956668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73700aec-b9c1-4e5e-b24c-92f907e73eb0 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:39.968056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316257b6-f40e-4a25-a070-4ac69cae188a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:39.969406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316257b6-f40e-4a25-a070-4ac69cae188a service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:39.970519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316257b6-f40e-4a25-a070-4ac69cae188a service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:39.971292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316257b6-f40e-4a25-a070-4ac69cae188a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:39.992033 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316257b6-f40e-4a25-a070-4ac69cae188a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:39.994334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316257b6-f40e-4a25-a070-4ac69cae188a service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:40.010798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 912/1867] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:40.014205 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dac821-d01b-45fc-864d-fcf212ad72f3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:40.017270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-44dac821-d01b-45fc-864d-fcf212ad72f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:40.033525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dac821-d01b-45fc-864d-fcf212ad72f3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:40.041434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:40.042642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:40.043273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:40.043757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] PATH_INFO: `/v3/users/efcc6b848f094ea7ba1dfcc656904da5` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:40.047831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efcc6b848f094ea7ba1dfcc656904da5)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:40.052007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:40.063355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:40.160981 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efcc6b848f094ea7ba1dfcc656904da5'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:40.196333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efcc6b848f094ea7ba1dfcc656904da5'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:40.215531 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efcc6b848f094ea7ba1dfcc656904da5'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:40.217064 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efcc6b848f094ea7ba1dfcc656904da5'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:40.231086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efcc6b848f094ea7ba1dfcc656904da5'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:40.245034 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 956/1868] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:40 2022] DELETE /identity/v3/users/efcc6b848f094ea7ba1dfcc656904da5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:40.250785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c970f-be85-4f89-b67c-439a20b89f14 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:40.252657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c970f-be85-4f89-b67c-439a20b89f14 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:40.253375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c970f-be85-4f89-b67c-439a20b89f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:40.283547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-465c970f-be85-4f89-b67c-439a20b89f14 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:40.284809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-465c970f-be85-4f89-b67c-439a20b89f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:40.314504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:40.384907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 913/1869] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:40.409954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:40.411533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:40.530132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12da3d-5d5f-4fb5-aedc-d8d7249294a4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:40.535489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af968e2-8f16-408f-b940-a527d2522ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:40.536587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af968e2-8f16-408f-b940-a527d2522ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:40.537047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af968e2-8f16-408f-b940-a527d2522ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:40.537352 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af968e2-8f16-408f-b940-a527d2522ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:40.550567 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af968e2-8f16-408f-b940-a527d2522ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:40.552287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af968e2-8f16-408f-b940-a527d2522ee1 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:40.566600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 957/1870] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:49:41.106364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c970f-be85-4f89-b67c-439a20b89f14 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:41.109285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-465c970f-be85-4f89-b67c-439a20b89f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:41.134819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c970f-be85-4f89-b67c-439a20b89f14 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:41.157199 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:41.159125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:41.162379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:41.162929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] PATH_INFO: `/v3/projects/eee0060807e441f58b892ff9d3845254` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:41.165825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee0060807e441f58b892ff9d3845254)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:41.169641 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:41.285552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee0060807e441f58b892ff9d3845254', 'request_id': 'req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:41.295552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee0060807e441f58b892ff9d3845254', 'request_id': 'req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:41.304695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee0060807e441f58b892ff9d3845254', 'request_id': 'req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:41.323208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] The token cache is being invalidate because project eee0060807e441f58b892ff9d3845254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:41.323514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:41.324392 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-39562b5c-afb0-4fe3-b2d4-818ddd4e189b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:41.327300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 914/1871] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:41 2022] DELETE /identity/v3/projects/eee0060807e441f58b892ff9d3845254 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:41.331261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af968e2-8f16-408f-b940-a527d2522ee1 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:41.345637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af968e2-8f16-408f-b940-a527d2522ee1 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:41.464958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cba5ba-c9a5-4891-9cd7-136a232d390c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:41.466576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cba5ba-c9a5-4891-9cd7-136a232d390c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:41.466871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cba5ba-c9a5-4891-9cd7-136a232d390c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:41.467154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cba5ba-c9a5-4891-9cd7-136a232d390c admin admin] PATH_INFO: `/v3/users/111d834e660a4ba68314fbeea8d37364` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:41.468745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cba5ba-c9a5-4891-9cd7-136a232d390c admin admin] RBAC: Authorizing `identity:delete_user(user_id=111d834e660a4ba68314fbeea8d37364)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:41.471635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cba5ba-c9a5-4891-9cd7-136a232d390c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:41.562604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:41.613912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-89cba5ba-c9a5-4891-9cd7-136a232d390c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '111d834e660a4ba68314fbeea8d37364'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:41.645514 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-89cba5ba-c9a5-4891-9cd7-136a232d390c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '111d834e660a4ba68314fbeea8d37364'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:41.659470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-89cba5ba-c9a5-4891-9cd7-136a232d390c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '111d834e660a4ba68314fbeea8d37364'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:41.660229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-89cba5ba-c9a5-4891-9cd7-136a232d390c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '111d834e660a4ba68314fbeea8d37364'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:41.671534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-89cba5ba-c9a5-4891-9cd7-136a232d390c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '111d834e660a4ba68314fbeea8d37364'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:41.690450 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 958/1872] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:41 2022] DELETE /identity/v3/users/111d834e660a4ba68314fbeea8d37364 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:41.924047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ac700-ffc2-44e1-9d22-64d5ce1f05ff None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:41.924371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ac700-ffc2-44e1-9d22-64d5ce1f05ff None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:41.924741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ac700-ffc2-44e1-9d22-64d5ce1f05ff None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:41.930276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 915/1873] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 20:49:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:49:41.984462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4c229-a8a4-4a47-80cd-26c2c8d9b59c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:41.985156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4c229-a8a4-4a47-80cd-26c2c8d9b59c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:41.986106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4c229-a8a4-4a47-80cd-26c2c8d9b59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:42.027479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-58b4c229-a8a4-4a47-80cd-26c2c8d9b59c None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:42.029474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-58b4c229-a8a4-4a47-80cd-26c2c8d9b59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:42.145330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ac700-ffc2-44e1-9d22-64d5ce1f05ff None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:42.147237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c24ac700-ffc2-44e1-9d22-64d5ce1f05ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:42.179269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 959/1874] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 20:49:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:42.237683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4c229-a8a4-4a47-80cd-26c2c8d9b59c None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:42.237970 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-58b4c229-a8a4-4a47-80cd-26c2c8d9b59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:42.258105 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4c229-a8a4-4a47-80cd-26c2c8d9b59c None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:42.265583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ae73e-c415-43ca-b801-1961bb728f9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:42.266237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ac700-ffc2-44e1-9d22-64d5ce1f05ff None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:42.266359 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ae73e-c415-43ca-b801-1961bb728f9a service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:42.266532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ae73e-c415-43ca-b801-1961bb728f9a service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:42.266859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ae73e-c415-43ca-b801-1961bb728f9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:42.273523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:42.274831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:42.275369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:42.275835 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] PATH_INFO: `/v3/projects/abe585184af940f7a997e3a212da46ba` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:42.277688 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe585184af940f7a997e3a212da46ba)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:42.279640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ae73e-c415-43ca-b801-1961bb728f9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:42.280515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:42.283806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ae73e-c415-43ca-b801-1961bb728f9a service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:42.335288 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 960/1875] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:49:42.388573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe585184af940f7a997e3a212da46ba', 'request_id': 'req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.392765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:42.419927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe585184af940f7a997e3a212da46ba', 'request_id': 'req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.431011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe585184af940f7a997e3a212da46ba', 'request_id': 'req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.450294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] The token cache is being invalidate because project abe585184af940f7a997e3a212da46ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:42.451393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.452180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.455207 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 916/1876] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:42 2022] DELETE /identity/v3/projects/abe585184af940f7a997e3a212da46ba => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:42.463318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ae73e-c415-43ca-b801-1961bb728f9a service nova] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:42.479394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ae73e-c415-43ca-b801-1961bb728f9a service nova] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:42.586211 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:42.588133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:42.604647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d83436-75b8-4413-8339-82e06ee4dd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:42.606978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d83436-75b8-4413-8339-82e06ee4dd60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:42.608446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d83436-75b8-4413-8339-82e06ee4dd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:42.608733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d83436-75b8-4413-8339-82e06ee4dd60 admin admin] PATH_INFO: `/v3/users/19f9d56cea8a4211b2fe3bbcccfeff7e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:42.610275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d83436-75b8-4413-8339-82e06ee4dd60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19f9d56cea8a4211b2fe3bbcccfeff7e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:42.612302 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d83436-75b8-4413-8339-82e06ee4dd60 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:42.616026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e69ff-ab8b-4faf-9d1e-51f80754e172 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:42.631789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:42.632704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:42.632990 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:42.633268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] PATH_INFO: `/v3/projects/8d5f50a377b14ca7a4daff8fe20d8ced` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:42.634824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d5f50a377b14ca7a4daff8fe20d8ced)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:42.637164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:42.684655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:42.723588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-12d83436-75b8-4413-8339-82e06ee4dd60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19f9d56cea8a4211b2fe3bbcccfeff7e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.765184 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-12d83436-75b8-4413-8339-82e06ee4dd60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19f9d56cea8a4211b2fe3bbcccfeff7e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.779332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d5f50a377b14ca7a4daff8fe20d8ced', 'request_id': 'req-74782988-b62b-4f8b-a9f2-f49df1e19438'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.786638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-12d83436-75b8-4413-8339-82e06ee4dd60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19f9d56cea8a4211b2fe3bbcccfeff7e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.790099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-12d83436-75b8-4413-8339-82e06ee4dd60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19f9d56cea8a4211b2fe3bbcccfeff7e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.791366 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d5f50a377b14ca7a4daff8fe20d8ced', 'request_id': 'req-74782988-b62b-4f8b-a9f2-f49df1e19438'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.804180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-12d83436-75b8-4413-8339-82e06ee4dd60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19f9d56cea8a4211b2fe3bbcccfeff7e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.810367 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d5f50a377b14ca7a4daff8fe20d8ced', 'request_id': 'req-74782988-b62b-4f8b-a9f2-f49df1e19438'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.818345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 961/1877] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:42 2022] DELETE /identity/v3/users/19f9d56cea8a4211b2fe3bbcccfeff7e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:42.835654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] The token cache is being invalidate because project 8d5f50a377b14ca7a4daff8fe20d8ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:42.835953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.837457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-74782988-b62b-4f8b-a9f2-f49df1e19438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:42.839361 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 917/1878] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:42 2022] DELETE /identity/v3/projects/8d5f50a377b14ca7a4daff8fe20d8ced => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:42.845201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b4712-ec5f-4eca-a00d-fa35ac05944f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:42.845943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b4712-ec5f-4eca-a00d-fa35ac05944f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:42.845943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b4712-ec5f-4eca-a00d-fa35ac05944f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:42.870623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-7f2b4712-ec5f-4eca-a00d-fa35ac05944f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:42.875336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-7f2b4712-ec5f-4eca-a00d-fa35ac05944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:42.971655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 962/1879] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:42.979750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337ca1-82dd-464d-9f94-9826ccbd3713 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:42.979915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337ca1-82dd-464d-9f94-9826ccbd3713 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:42.980235 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337ca1-82dd-464d-9f94-9826ccbd3713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.009330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-32337ca1-82dd-464d-9f94-9826ccbd3713 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:43.010683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-32337ca1-82dd-464d-9f94-9826ccbd3713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:43.036317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 918/1880] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:43.041699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b4712-ec5f-4eca-a00d-fa35ac05944f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.084296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b4712-ec5f-4eca-a00d-fa35ac05944f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.098677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb1feb-3a89-4b3a-b442-db4ffa2fa8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.100209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb1feb-3a89-4b3a-b442-db4ffa2fa8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.101126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb1feb-3a89-4b3a-b442-db4ffa2fa8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.101868 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb1feb-3a89-4b3a-b442-db4ffa2fa8f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.103416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb1feb-3a89-4b3a-b442-db4ffa2fa8f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:43.104194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb1feb-3a89-4b3a-b442-db4ffa2fa8f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:43.112525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb1feb-3a89-4b3a-b442-db4ffa2fa8f6 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:43.152258 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337ca1-82dd-464d-9f94-9826ccbd3713 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.153417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 963/1881] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:43.162604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337ca1-82dd-464d-9f94-9826ccbd3713 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.167593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628dd77-1de0-4622-9efc-07132b04b988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.168358 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628dd77-1de0-4622-9efc-07132b04b988 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.168839 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628dd77-1de0-4622-9efc-07132b04b988 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.169212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628dd77-1de0-4622-9efc-07132b04b988 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.170006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628dd77-1de0-4622-9efc-07132b04b988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:43.172490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628dd77-1de0-4622-9efc-07132b04b988 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:43.212597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 919/1882] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:43 2022] POST /identity/v3/projects => generated 409 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:43.216966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb1feb-3a89-4b3a-b442-db4ffa2fa8f6 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.258954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb1feb-3a89-4b3a-b442-db4ffa2fa8f6 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.337051 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3658e7-db09-4dba-8651-63e91bf56402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.337926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3658e7-db09-4dba-8651-63e91bf56402 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.338006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3658e7-db09-4dba-8651-63e91bf56402 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.338747 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3658e7-db09-4dba-8651-63e91bf56402 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.339480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3658e7-db09-4dba-8651-63e91bf56402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:43.341447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3658e7-db09-4dba-8651-63e91bf56402 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:43.395010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 964/1883] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:43 2022] POST /identity/v3/users => generated 305 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:43.395946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628dd77-1de0-4622-9efc-07132b04b988 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.417703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628dd77-1de0-4622-9efc-07132b04b988 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.425098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a265d-9aa0-46db-9b04-faafece7dbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.425977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a265d-9aa0-46db-9b04-faafece7dbae admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.426323 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a265d-9aa0-46db-9b04-faafece7dbae admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.426961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a265d-9aa0-46db-9b04-faafece7dbae admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.427883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a265d-9aa0-46db-9b04-faafece7dbae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:43.429864 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a265d-9aa0-46db-9b04-faafece7dbae admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:43.445974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 920/1884] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:49:43.451560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3658e7-db09-4dba-8651-63e91bf56402 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.474151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3658e7-db09-4dba-8651-63e91bf56402 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.481520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a265d-9aa0-46db-9b04-faafece7dbae admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.483859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-231a265d-9aa0-46db-9b04-faafece7dbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:43.488344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383d239-c86b-4214-b7d2-5c227c3fe834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.489555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383d239-c86b-4214-b7d2-5c227c3fe834 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.490079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383d239-c86b-4214-b7d2-5c227c3fe834 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.490588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383d239-c86b-4214-b7d2-5c227c3fe834 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.493300 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383d239-c86b-4214-b7d2-5c227c3fe834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:43.493461 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383d239-c86b-4214-b7d2-5c227c3fe834 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:43.519880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 965/1885] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:43.520836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383d239-c86b-4214-b7d2-5c227c3fe834 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.537607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a265d-9aa0-46db-9b04-faafece7dbae admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.539637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383d239-c86b-4214-b7d2-5c227c3fe834 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.549860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.551007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.551463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.552154 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] PATH_INFO: `/v3/projects/ce1087799293447484f34833f6ce60ee` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.554918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285696bb-b1f8-42ba-bbd1-a43043e58443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.555825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285696bb-b1f8-42ba-bbd1-a43043e58443 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.556108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285696bb-b1f8-42ba-bbd1-a43043e58443 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.556389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285696bb-b1f8-42ba-bbd1-a43043e58443 admin admin] PATH_INFO: `/v3/projects/5fc379c2164c467ebc4379166ac862d3/users/15ecd34476e14193b77d7341dd907578/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.557020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce1087799293447484f34833f6ce60ee)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:43.559119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:43.565436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:43.603044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285696bb-b1f8-42ba-bbd1-a43043e58443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc379c2164c467ebc4379166ac862d3, user_id=15ecd34476e14193b77d7341dd907578, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:43.608937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285696bb-b1f8-42ba-bbd1-a43043e58443 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:43.629840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce1087799293447484f34833f6ce60ee', 'request_id': 'req-9cdc0c03-d631-4576-bd5c-e5f388a1903d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:43.636157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 966/1886] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:43 2022] PUT /identity/v3/projects/5fc379c2164c467ebc4379166ac862d3/users/15ecd34476e14193b77d7341dd907578/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:43.644794 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce1087799293447484f34833f6ce60ee', 'request_id': 'req-9cdc0c03-d631-4576-bd5c-e5f388a1903d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:43.645260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c6063-801e-453c-98ed-309e0d45c374 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.645853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c6063-801e-453c-98ed-309e0d45c374 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.646351 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c6063-801e-453c-98ed-309e0d45c374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.660076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce1087799293447484f34833f6ce60ee', 'request_id': 'req-9cdc0c03-d631-4576-bd5c-e5f388a1903d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:43.682236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] The token cache is being invalidate because project ce1087799293447484f34833f6ce60ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:43.682646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:43.684490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9cdc0c03-d631-4576-bd5c-e5f388a1903d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:43.686875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 921/1887] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:43 2022] DELETE /identity/v3/projects/ce1087799293447484f34833f6ce60ee => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:43.693678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21050f0f-fef3-4af1-9b74-4276237def27 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.694147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21050f0f-fef3-4af1-9b74-4276237def27 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.694460 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21050f0f-fef3-4af1-9b74-4276237def27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.697010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-af8c6063-801e-453c-98ed-309e0d45c374 None None] MFA Rules not processed for user `15ecd34476e14193b77d7341dd907578`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:43.699301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-af8c6063-801e-453c-98ed-309e0d45c374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:43.709179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=79,keystone:DELETE=17,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:43.717945 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-21050f0f-fef3-4af1-9b74-4276237def27 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:43.721266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-21050f0f-fef3-4af1-9b74-4276237def27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:43.809047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 967/1888] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:43.811052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 922/1889] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:43.813506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c6063-801e-453c-98ed-309e0d45c374 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.814812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-af8c6063-801e-453c-98ed-309e0d45c374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:43.818721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fafa85-c873-4e2e-952c-133ea37cba1d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.819011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fafa85-c873-4e2e-952c-133ea37cba1d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.819291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fafa85-c873-4e2e-952c-133ea37cba1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.828618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c6063-801e-453c-98ed-309e0d45c374 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.833585 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a64397-1fee-44a7-b5fa-354dad6e2a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.834251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a64397-1fee-44a7-b5fa-354dad6e2a84 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.834566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a64397-1fee-44a7-b5fa-354dad6e2a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.834793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a64397-1fee-44a7-b5fa-354dad6e2a84 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.835482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a64397-1fee-44a7-b5fa-354dad6e2a84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:43.837425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a64397-1fee-44a7-b5fa-354dad6e2a84 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:43.837738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-f2fafa85-c873-4e2e-952c-133ea37cba1d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:43.838928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-f2fafa85-c873-4e2e-952c-133ea37cba1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:43.847906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 923/1890] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:43.852317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fafa85-c873-4e2e-952c-133ea37cba1d None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.862798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fafa85-c873-4e2e-952c-133ea37cba1d None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.867340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c047c-624c-4c8f-85d1-af845f38bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.868000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c047c-624c-4c8f-85d1-af845f38bb9e admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.868407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c047c-624c-4c8f-85d1-af845f38bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.868559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c047c-624c-4c8f-85d1-af845f38bb9e admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.869286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c047c-624c-4c8f-85d1-af845f38bb9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:43.869625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c047c-624c-4c8f-85d1-af845f38bb9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:43.873298 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 968/1891] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:43 2022] POST /identity/v3/projects => generated 409 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:49:43.873468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c047c-624c-4c8f-85d1-af845f38bb9e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:43.874414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a64397-1fee-44a7-b5fa-354dad6e2a84 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.885127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 924/1892] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:49:43.889743 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c047c-624c-4c8f-85d1-af845f38bb9e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.899699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c047c-624c-4c8f-85d1-af845f38bb9e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.940870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a64397-1fee-44a7-b5fa-354dad6e2a84 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.947257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35a70d-d05f-43f6-9291-cbe53b26ea48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.948420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35a70d-d05f-43f6-9291-cbe53b26ea48 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.948943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35a70d-d05f-43f6-9291-cbe53b26ea48 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.949317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35a70d-d05f-43f6-9291-cbe53b26ea48 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.950180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35a70d-d05f-43f6-9291-cbe53b26ea48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:43.952296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35a70d-d05f-43f6-9291-cbe53b26ea48 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:43.979244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 969/1893] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:43 2022] POST /identity/v3/users => generated 311 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:43.983503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35a70d-d05f-43f6-9291-cbe53b26ea48 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:43.986091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdad992-eeed-45ee-94fc-fc8cd3ff29c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.986777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdad992-eeed-45ee-94fc-fc8cd3ff29c3 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:43.987066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdad992-eeed-45ee-94fc-fc8cd3ff29c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:43.987348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdad992-eeed-45ee-94fc-fc8cd3ff29c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:43.988296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdad992-eeed-45ee-94fc-fc8cd3ff29c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:43.990171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdad992-eeed-45ee-94fc-fc8cd3ff29c3 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:43.994050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35a70d-d05f-43f6-9291-cbe53b26ea48 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:43.999025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a4bdd-9f7e-406a-af78-cdb86c77e0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:43.999820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a4bdd-9f7e-406a-af78-cdb86c77e0fd admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.000205 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a4bdd-9f7e-406a-af78-cdb86c77e0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.000551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a4bdd-9f7e-406a-af78-cdb86c77e0fd admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.001415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a4bdd-9f7e-406a-af78-cdb86c77e0fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.003160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a4bdd-9f7e-406a-af78-cdb86c77e0fd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.015521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 970/1894] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:49:44.020323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a4bdd-9f7e-406a-af78-cdb86c77e0fd admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.026213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 925/1895] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:43 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:49:44.031030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdad992-eeed-45ee-94fc-fc8cd3ff29c3 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.035700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a4bdd-9f7e-406a-af78-cdb86c77e0fd admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.067320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:44.106308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdad992-eeed-45ee-94fc-fc8cd3ff29c3 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.111256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba625f-bdb5-406b-8836-22cd4abe7ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.111712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c7afb-d16e-4383-8735-d47b9ea0642f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.111960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba625f-bdb5-406b-8836-22cd4abe7ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.112249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba625f-bdb5-406b-8836-22cd4abe7ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.112537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba625f-bdb5-406b-8836-22cd4abe7ef2 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.112914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c7afb-d16e-4383-8735-d47b9ea0642f admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.113212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c7afb-d16e-4383-8735-d47b9ea0642f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.113316 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba625f-bdb5-406b-8836-22cd4abe7ef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.113669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c7afb-d16e-4383-8735-d47b9ea0642f admin admin] PATH_INFO: `/v3/projects/8bdec1ddbee7439fbaff85fc3c0d136c/users/34d7839ef5f64fb9b09b82ded4596a2b/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.115490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba625f-bdb5-406b-8836-22cd4abe7ef2 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.132159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c7afb-d16e-4383-8735-d47b9ea0642f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdec1ddbee7439fbaff85fc3c0d136c, user_id=34d7839ef5f64fb9b09b82ded4596a2b, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.134998 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c7afb-d16e-4383-8735-d47b9ea0642f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.139031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 926/1896] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:44 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:44.144070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba625f-bdb5-406b-8836-22cd4abe7ef2 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.149603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:44.154342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 971/1897] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:44 2022] PUT /identity/v3/projects/8bdec1ddbee7439fbaff85fc3c0d136c/users/34d7839ef5f64fb9b09b82ded4596a2b/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:44.156293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba625f-bdb5-406b-8836-22cd4abe7ef2 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.158863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c7afb-d16e-4383-8735-d47b9ea0642f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.215617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5024a6-c56f-4c32-a359-1facf9ce6837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.216300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5024a6-c56f-4c32-a359-1facf9ce6837 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.217053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5024a6-c56f-4c32-a359-1facf9ce6837 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.217053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5024a6-c56f-4c32-a359-1facf9ce6837 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.217862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5024a6-c56f-4c32-a359-1facf9ce6837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.219469 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5024a6-c56f-4c32-a359-1facf9ce6837 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.221425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c7afb-d16e-4383-8735-d47b9ea0642f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.226578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29257c66-cde3-440c-8e2e-fadb15044fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.227242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29257c66-cde3-440c-8e2e-fadb15044fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.227684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29257c66-cde3-440c-8e2e-fadb15044fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.227961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29257c66-cde3-440c-8e2e-fadb15044fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.228685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29257c66-cde3-440c-8e2e-fadb15044fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.230289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29257c66-cde3-440c-8e2e-fadb15044fe5 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.230632 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 927/1898] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:49:44.234605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5024a6-c56f-4c32-a359-1facf9ce6837 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.241417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 972/1899] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:49:44.245045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5024a6-c56f-4c32-a359-1facf9ce6837 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.245417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29257c66-cde3-440c-8e2e-fadb15044fe5 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.249820 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a846d1-0714-46b1-8f54-d5e597f19417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.250486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a846d1-0714-46b1-8f54-d5e597f19417 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.250775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a846d1-0714-46b1-8f54-d5e597f19417 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.251275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a846d1-0714-46b1-8f54-d5e597f19417 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.252015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a846d1-0714-46b1-8f54-d5e597f19417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.253636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a846d1-0714-46b1-8f54-d5e597f19417 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.255260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29257c66-cde3-440c-8e2e-fadb15044fe5 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.259952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ae51d-5489-4b37-ae4c-622ba1f86f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.260827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ae51d-5489-4b37-ae4c-622ba1f86f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.261116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ae51d-5489-4b37-ae4c-622ba1f86f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.261430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ae51d-5489-4b37-ae4c-622ba1f86f42 admin admin] PATH_INFO: `/v3/projects/8bdec1ddbee7439fbaff85fc3c0d136c/users/34d7839ef5f64fb9b09b82ded4596a2b/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.263433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ae51d-5489-4b37-ae4c-622ba1f86f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdec1ddbee7439fbaff85fc3c0d136c, user_id=34d7839ef5f64fb9b09b82ded4596a2b, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.266182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ae51d-5489-4b37-ae4c-622ba1f86f42 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.267650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 928/1900] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:49:44.268330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a846d1-0714-46b1-8f54-d5e597f19417 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.277261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 973/1901] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:44 2022] PUT /identity/v3/projects/8bdec1ddbee7439fbaff85fc3c0d136c/users/34d7839ef5f64fb9b09b82ded4596a2b/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:44.278462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a846d1-0714-46b1-8f54-d5e597f19417 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.288988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adfbb7-f986-4cd9-8080-8fd4c0d35b44 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.289283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adfbb7-f986-4cd9-8080-8fd4c0d35b44 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.289603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adfbb7-f986-4cd9-8080-8fd4c0d35b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.318439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-78adfbb7-f986-4cd9-8080-8fd4c0d35b44 None None] MFA Rules not processed for user `34d7839ef5f64fb9b09b82ded4596a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:44.319801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-78adfbb7-f986-4cd9-8080-8fd4c0d35b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:44.343809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383d6bd-19d7-4331-b03f-60c42ded389f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.344732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383d6bd-19d7-4331-b03f-60c42ded389f admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.345176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383d6bd-19d7-4331-b03f-60c42ded389f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.345492 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383d6bd-19d7-4331-b03f-60c42ded389f admin admin] PATH_INFO: `/v3/projects/8872ff4e7dc741eb91ae535754ad94ac/users/2d14c555c0e04477b269adb6b9a206a8/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.362108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383d6bd-19d7-4331-b03f-60c42ded389f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8872ff4e7dc741eb91ae535754ad94ac, user_id=2d14c555c0e04477b269adb6b9a206a8, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.365256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383d6bd-19d7-4331-b03f-60c42ded389f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.393148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 929/1902] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:44 2022] PUT /identity/v3/projects/8872ff4e7dc741eb91ae535754ad94ac/users/2d14c555c0e04477b269adb6b9a206a8/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:44.404511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d0b37-c4a7-4547-8016-21d765fcd39d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.405155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d0b37-c4a7-4547-8016-21d765fcd39d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.405155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d0b37-c4a7-4547-8016-21d765fcd39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.406017 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 974/1903] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:44 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:44.419847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adfbb7-f986-4cd9-8080-8fd4c0d35b44 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.423100 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-78adfbb7-f986-4cd9-8080-8fd4c0d35b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:44.440488 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ce4d0b37-c4a7-4547-8016-21d765fcd39d None None] MFA Rules not processed for user `2d14c555c0e04477b269adb6b9a206a8`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:44.442354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ce4d0b37-c4a7-4547-8016-21d765fcd39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:44.462265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:44.503331 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adfbb7-f986-4cd9-8080-8fd4c0d35b44 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.529169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:44.534470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 930/1904] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:44.539457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d0b37-c4a7-4547-8016-21d765fcd39d None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.564465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc0701-9d1f-4cdb-ad67-3458ab24f876 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.565156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc0701-9d1f-4cdb-ad67-3458ab24f876 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.565465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc0701-9d1f-4cdb-ad67-3458ab24f876 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.565750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc0701-9d1f-4cdb-ad67-3458ab24f876 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.616543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d0b37-c4a7-4547-8016-21d765fcd39d None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.621866 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a595a4-648e-47db-bfb9-36a733526638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.622551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a595a4-648e-47db-bfb9-36a733526638 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.622834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a595a4-648e-47db-bfb9-36a733526638 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.623107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a595a4-648e-47db-bfb9-36a733526638 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.623807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a595a4-648e-47db-bfb9-36a733526638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.625770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a595a4-648e-47db-bfb9-36a733526638 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.662222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 931/1905] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:44 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:44.663313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a595a4-648e-47db-bfb9-36a733526638 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.668412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc0701-9d1f-4cdb-ad67-3458ab24f876 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.670307 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc0701-9d1f-4cdb-ad67-3458ab24f876 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.692333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 975/1906] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:44 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:44.749289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a595a4-648e-47db-bfb9-36a733526638 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.755792 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb88e0-4a07-464a-8387-42b83707025a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.758516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb88e0-4a07-464a-8387-42b83707025a admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.758876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb88e0-4a07-464a-8387-42b83707025a admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.759317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb88e0-4a07-464a-8387-42b83707025a admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.760091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb88e0-4a07-464a-8387-42b83707025a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.762151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb88e0-4a07-464a-8387-42b83707025a admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.810082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc0701-9d1f-4cdb-ad67-3458ab24f876 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.820556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 932/1907] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:44 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:44.825503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc0701-9d1f-4cdb-ad67-3458ab24f876 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.831007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd6e6a-d9b6-4a56-a6ec-d21981a0a493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.831808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd6e6a-d9b6-4a56-a6ec-d21981a0a493 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.832198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd6e6a-d9b6-4a56-a6ec-d21981a0a493 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.832579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd6e6a-d9b6-4a56-a6ec-d21981a0a493 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.833459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd6e6a-d9b6-4a56-a6ec-d21981a0a493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.835321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd6e6a-d9b6-4a56-a6ec-d21981a0a493 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.852666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 976/1908] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:44.857036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb88e0-4a07-464a-8387-42b83707025a admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.882039 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb88e0-4a07-464a-8387-42b83707025a admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.888812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea524b-c9f5-4e9f-83fd-0068a937e16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:44.889966 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea524b-c9f5-4e9f-83fd-0068a937e16b admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:44.890973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea524b-c9f5-4e9f-83fd-0068a937e16b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:44.891709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea524b-c9f5-4e9f-83fd-0068a937e16b admin admin] PATH_INFO: `/v3/projects/503c15bde2ca4748a6cd1a3785671c11/users/64ddedb7c02a4f4fa890b4bb2b3f6e14/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:44.912365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea524b-c9f5-4e9f-83fd-0068a937e16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=503c15bde2ca4748a6cd1a3785671c11, user_id=64ddedb7c02a4f4fa890b4bb2b3f6e14, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:44.915386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea524b-c9f5-4e9f-83fd-0068a937e16b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:44.937050 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 933/1909] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:44 2022] PUT /identity/v3/projects/503c15bde2ca4748a6cd1a3785671c11/users/64ddedb7c02a4f4fa890b4bb2b3f6e14/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:44.938339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd6e6a-d9b6-4a56-a6ec-d21981a0a493 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:44.952492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd6e6a-d9b6-4a56-a6ec-d21981a0a493 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:44.971612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea524b-c9f5-4e9f-83fd-0068a937e16b admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:45.027665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24dd24-2214-414c-ad93-c897489c65bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:45.028529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24dd24-2214-414c-ad93-c897489c65bd admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:45.028911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24dd24-2214-414c-ad93-c897489c65bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:45.029412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24dd24-2214-414c-ad93-c897489c65bd admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:45.030600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24dd24-2214-414c-ad93-c897489c65bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:45.032945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24dd24-2214-414c-ad93-c897489c65bd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:45.051626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 977/1910] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:45.052246 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea524b-c9f5-4e9f-83fd-0068a937e16b admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:45.052914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24dd24-2214-414c-ad93-c897489c65bd admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:45.063480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24dd24-2214-414c-ad93-c897489c65bd admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:45.065621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20042b62-f455-43a8-8528-6f3eb89953ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:45.066289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20042b62-f455-43a8-8528-6f3eb89953ac service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:45.066592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20042b62-f455-43a8-8528-6f3eb89953ac service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:45.066885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20042b62-f455-43a8-8528-6f3eb89953ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:45.068817 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09f4ed-8307-4a23-b9c8-fc06dea7692f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:45.069732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09f4ed-8307-4a23-b9c8-fc06dea7692f admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:45.070045 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09f4ed-8307-4a23-b9c8-fc06dea7692f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:45.070319 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09f4ed-8307-4a23-b9c8-fc06dea7692f admin admin] PATH_INFO: `/v3/projects/503c15bde2ca4748a6cd1a3785671c11/users/64ddedb7c02a4f4fa890b4bb2b3f6e14/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:45.071419 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:45.072317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09f4ed-8307-4a23-b9c8-fc06dea7692f admin admin] RBAC: Authorizing `identity:create_grant(project_id=503c15bde2ca4748a6cd1a3785671c11, user_id=64ddedb7c02a4f4fa890b4bb2b3f6e14, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:45.075104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09f4ed-8307-4a23-b9c8-fc06dea7692f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:45.082741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:45.090362 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 978/1911] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:45 2022] PUT /identity/v3/projects/503c15bde2ca4748a6cd1a3785671c11/users/64ddedb7c02a4f4fa890b4bb2b3f6e14/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:45.101468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc83b7f-f8d3-4afd-98b1-3e98785f79cc None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:45.101758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc83b7f-f8d3-4afd-98b1-3e98785f79cc None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:45.102040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc83b7f-f8d3-4afd-98b1-3e98785f79cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:45.133979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20042b62-f455-43a8-8528-6f3eb89953ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:45.134173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-1dc83b7f-f8d3-4afd-98b1-3e98785f79cc None None] MFA Rules not processed for user `64ddedb7c02a4f4fa890b4bb2b3f6e14`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:45.135230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1dc83b7f-f8d3-4afd-98b1-3e98785f79cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:45.135645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20042b62-f455-43a8-8528-6f3eb89953ac service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:45.145322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 934/1912] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:44 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:45.206316 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 979/1913] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:45.218213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20042b62-f455-43a8-8528-6f3eb89953ac service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:45.219608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-20042b62-f455-43a8-8528-6f3eb89953ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:45.231663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20042b62-f455-43a8-8528-6f3eb89953ac service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:45.284463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82080e9-1e7a-4fbb-b34e-f68bb8f216ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:45.285350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82080e9-1e7a-4fbb-b34e-f68bb8f216ed service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:45.285764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82080e9-1e7a-4fbb-b34e-f68bb8f216ed service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:45.286151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82080e9-1e7a-4fbb-b34e-f68bb8f216ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:45.346157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82080e9-1e7a-4fbb-b34e-f68bb8f216ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:45.347704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82080e9-1e7a-4fbb-b34e-f68bb8f216ed service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:45.358199 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 935/1914] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:53.321480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc83b7f-f8d3-4afd-98b1-3e98785f79cc None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:53.430049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc83b7f-f8d3-4afd-98b1-3e98785f79cc None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:53.435225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:53.436153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:53.436435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:53.436717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] PATH_INFO: `/v3/users/2d14c555c0e04477b269adb6b9a206a8` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:53.438003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d14c555c0e04477b269adb6b9a206a8)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:53.440734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:53.490668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d14c555c0e04477b269adb6b9a206a8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:53.503237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d14c555c0e04477b269adb6b9a206a8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:53.511251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d14c555c0e04477b269adb6b9a206a8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:53.511919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d14c555c0e04477b269adb6b9a206a8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:53.519595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d14c555c0e04477b269adb6b9a206a8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:53.525729 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 980/1915] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:53 2022] DELETE /identity/v3/users/2d14c555c0e04477b269adb6b9a206a8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:53.530381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee8c03-9efb-43ec-9432-213183fd8754 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:53.530868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee8c03-9efb-43ec-9432-213183fd8754 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:53.530956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee8c03-9efb-43ec-9432-213183fd8754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:53.548903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-baee8c03-9efb-43ec-9432-213183fd8754 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:53.550221 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-baee8c03-9efb-43ec-9432-213183fd8754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:53.602343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:53.611827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 936/1916] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:53.622194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:53.623820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:53.632267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:53.696176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd7fc4-6caf-4798-8309-2f1185d9b2dc admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:53.699873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72b3ea-e313-405e-af33-33dd0c8663ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:53.700589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72b3ea-e313-405e-af33-33dd0c8663ed service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:53.700939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72b3ea-e313-405e-af33-33dd0c8663ed service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:53.701278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72b3ea-e313-405e-af33-33dd0c8663ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:53.713207 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72b3ea-e313-405e-af33-33dd0c8663ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:53.714760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72b3ea-e313-405e-af33-33dd0c8663ed service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:53.726275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 981/1917] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:54.032778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee8c03-9efb-43ec-9432-213183fd8754 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:54.034360 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-baee8c03-9efb-43ec-9432-213183fd8754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:54.047500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee8c03-9efb-43ec-9432-213183fd8754 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:54.053317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:54.054341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:54.054720 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:54.055243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] PATH_INFO: `/v3/projects/8872ff4e7dc741eb91ae535754ad94ac` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:54.056540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8872ff4e7dc741eb91ae535754ad94ac)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:54.058833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:54.116808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8872ff4e7dc741eb91ae535754ad94ac', 'request_id': 'req-9c392477-d8b4-4297-b1fe-81760b9e6ce2'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:54.123832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8872ff4e7dc741eb91ae535754ad94ac', 'request_id': 'req-9c392477-d8b4-4297-b1fe-81760b9e6ce2'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:54.130937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8872ff4e7dc741eb91ae535754ad94ac', 'request_id': 'req-9c392477-d8b4-4297-b1fe-81760b9e6ce2'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:54.143965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] The token cache is being invalidate because project 8872ff4e7dc741eb91ae535754ad94ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:54.144260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:54.144914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:54.146523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 937/1918] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:54 2022] DELETE /identity/v3/projects/8872ff4e7dc741eb91ae535754ad94ac => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:54.150669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72b3ea-e313-405e-af33-33dd0c8663ed service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:54.160784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72b3ea-e313-405e-af33-33dd0c8663ed service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:54.232636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:54.233549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:54.233964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:54.234269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] PATH_INFO: `/v3/users/64ddedb7c02a4f4fa890b4bb2b3f6e14` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:54.236287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ddedb7c02a4f4fa890b4bb2b3f6e14)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:54.239013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:54.296399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ddedb7c02a4f4fa890b4bb2b3f6e14'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:54.311153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ddedb7c02a4f4fa890b4bb2b3f6e14'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:54.320810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ddedb7c02a4f4fa890b4bb2b3f6e14'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:54.321705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ddedb7c02a4f4fa890b4bb2b3f6e14'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:54.330833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ddedb7c02a4f4fa890b4bb2b3f6e14'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:54.337735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 982/1919] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:54 2022] DELETE /identity/v3/users/64ddedb7c02a4f4fa890b4bb2b3f6e14 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:55.270803 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:55.272254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:55.287032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c392477-d8b4-4297-b1fe-81760b9e6ce2 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:55.347756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aefa9f-d862-4c2c-bfb2-38bdc47cb605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:55.348874 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aefa9f-d862-4c2c-bfb2-38bdc47cb605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:55.349280 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aefa9f-d862-4c2c-bfb2-38bdc47cb605 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:55.349650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aefa9f-d862-4c2c-bfb2-38bdc47cb605 admin admin] PATH_INFO: `/v3/users/d61e7f6848cd4efc919b1b360a1600d1` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:55.351150 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aefa9f-d862-4c2c-bfb2-38bdc47cb605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d61e7f6848cd4efc919b1b360a1600d1)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:55.353318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aefa9f-d862-4c2c-bfb2-38bdc47cb605 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:55.370560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:55.372090 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:55.388469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e069eb-bd9f-4fdc-aa32-aa06e89444fc admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:55.395647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:55.396725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:55.397115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:55.397479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] PATH_INFO: `/v3/projects/503c15bde2ca4748a6cd1a3785671c11` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:55.398914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=503c15bde2ca4748a6cd1a3785671c11)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:55.401219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:55.414676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73aefa9f-d862-4c2c-bfb2-38bdc47cb605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd61e7f6848cd4efc919b1b360a1600d1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:55.422063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73aefa9f-d862-4c2c-bfb2-38bdc47cb605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd61e7f6848cd4efc919b1b360a1600d1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:55.427152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:55.429193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73aefa9f-d862-4c2c-bfb2-38bdc47cb605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd61e7f6848cd4efc919b1b360a1600d1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:55.443498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73aefa9f-d862-4c2c-bfb2-38bdc47cb605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd61e7f6848cd4efc919b1b360a1600d1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:55.444200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-73aefa9f-d862-4c2c-bfb2-38bdc47cb605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd61e7f6848cd4efc919b1b360a1600d1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:55.450248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 938/1920] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:55 2022] DELETE /identity/v3/users/d61e7f6848cd4efc919b1b360a1600d1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:55.453696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '503c15bde2ca4748a6cd1a3785671c11', 'request_id': 'req-44d86cab-aa58-4ace-83e6-813b0be228fd'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:55.455483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde38f85-d733-424d-a0bf-b76af7943bae None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:55.456016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde38f85-d733-424d-a0bf-b76af7943bae None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:55.456304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde38f85-d733-424d-a0bf-b76af7943bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:55.462405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '503c15bde2ca4748a6cd1a3785671c11', 'request_id': 'req-44d86cab-aa58-4ace-83e6-813b0be228fd'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:55.462644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:55.469203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '503c15bde2ca4748a6cd1a3785671c11', 'request_id': 'req-44d86cab-aa58-4ace-83e6-813b0be228fd'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:55.477740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-fde38f85-d733-424d-a0bf-b76af7943bae None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:55.478880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-fde38f85-d733-424d-a0bf-b76af7943bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:55.481808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] The token cache is being invalidate because project 503c15bde2ca4748a6cd1a3785671c11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:55.482297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:55.483456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-44d86cab-aa58-4ace-83e6-813b0be228fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:55.485775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 983/1921] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:55 2022] DELETE /identity/v3/projects/503c15bde2ca4748a6cd1a3785671c11 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:55.491529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d310fa3-9034-48e1-abdf-c8ceb68fab9f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:55.492191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d310fa3-9034-48e1-abdf-c8ceb68fab9f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:55.492774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d310fa3-9034-48e1-abdf-c8ceb68fab9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:55.512030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-8d310fa3-9034-48e1-abdf-c8ceb68fab9f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:55.513296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8d310fa3-9034-48e1-abdf-c8ceb68fab9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:55.543173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 939/1922] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:55.553721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde38f85-d733-424d-a0bf-b76af7943bae None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:55.555174 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-fde38f85-d733-424d-a0bf-b76af7943bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:55.596331 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 984/1923] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:55.599922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569e8e9-7a49-4d24-a760-4d928e203aa0 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:55.600218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569e8e9-7a49-4d24-a760-4d928e203aa0 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:55.601452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569e8e9-7a49-4d24-a760-4d928e203aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:55.628784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e569e8e9-7a49-4d24-a760-4d928e203aa0 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:55.630349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e569e8e9-7a49-4d24-a760-4d928e203aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:55.637177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde38f85-d733-424d-a0bf-b76af7943bae None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:55.642701 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588e7ba-7666-416c-8a0a-ba2bd44bae14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:55.643369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e7ba-7666-416c-8a0a-ba2bd44bae14 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:55.643867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e7ba-7666-416c-8a0a-ba2bd44bae14 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:55.644257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e7ba-7666-416c-8a0a-ba2bd44bae14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:55.652270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 985/1924] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:49:55.655410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569e8e9-7a49-4d24-a760-4d928e203aa0 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:55.658896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588e7ba-7666-416c-8a0a-ba2bd44bae14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:55.660832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588e7ba-7666-416c-8a0a-ba2bd44bae14 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:55.675661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 940/1925] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:55.679255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569e8e9-7a49-4d24-a760-4d928e203aa0 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:55.687666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c52d2-dda4-41d9-a643-31eb60cfbb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:55.688704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c52d2-dda4-41d9-a643-31eb60cfbb27 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:55.688879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c52d2-dda4-41d9-a643-31eb60cfbb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:55.689155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c52d2-dda4-41d9-a643-31eb60cfbb27 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:55.690139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c52d2-dda4-41d9-a643-31eb60cfbb27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:55.690504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c52d2-dda4-41d9-a643-31eb60cfbb27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:55.695275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c52d2-dda4-41d9-a643-31eb60cfbb27 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:55.713152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 986/1926] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:55.714399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588e7ba-7666-416c-8a0a-ba2bd44bae14 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:55.743109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588e7ba-7666-416c-8a0a-ba2bd44bae14 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:55.758285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae16363-eaa0-4ba4-849c-a3e85ca14167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:55.759334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae16363-eaa0-4ba4-849c-a3e85ca14167 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:55.759927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae16363-eaa0-4ba4-849c-a3e85ca14167 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:55.760368 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae16363-eaa0-4ba4-849c-a3e85ca14167 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:55.761221 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae16363-eaa0-4ba4-849c-a3e85ca14167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:55.763340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae16363-eaa0-4ba4-849c-a3e85ca14167 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:55.801611 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 941/1927] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:55 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:55.806427 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c52d2-dda4-41d9-a643-31eb60cfbb27 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:55.819696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c52d2-dda4-41d9-a643-31eb60cfbb27 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:55.942566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e235ef-5141-41f4-be89-4a34d975b488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:55.943494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e235ef-5141-41f4-be89-4a34d975b488 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:55.944476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e235ef-5141-41f4-be89-4a34d975b488 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:55.944774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e235ef-5141-41f4-be89-4a34d975b488 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:55.946036 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e235ef-5141-41f4-be89-4a34d975b488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:55.948928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e235ef-5141-41f4-be89-4a34d975b488 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:56.005746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 987/1928] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:55 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:56.009216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae16363-eaa0-4ba4-849c-a3e85ca14167 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:56.023603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae16363-eaa0-4ba4-849c-a3e85ca14167 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:56.037764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd7421-9675-4aea-8c36-f11059eaf719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:56.039184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd7421-9675-4aea-8c36-f11059eaf719 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:56.039650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd7421-9675-4aea-8c36-f11059eaf719 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:56.041401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd7421-9675-4aea-8c36-f11059eaf719 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:56.043547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd7421-9675-4aea-8c36-f11059eaf719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:56.050206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd7421-9675-4aea-8c36-f11059eaf719 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:56.072230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 942/1929] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:56.073793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e235ef-5141-41f4-be89-4a34d975b488 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:56.091591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e235ef-5141-41f4-be89-4a34d975b488 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:56.098385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d06bcc-c47f-49e9-8ab9-f39f7a004443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:56.099916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d06bcc-c47f-49e9-8ab9-f39f7a004443 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:56.100049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d06bcc-c47f-49e9-8ab9-f39f7a004443 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:56.100334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d06bcc-c47f-49e9-8ab9-f39f7a004443 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:56.101071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d06bcc-c47f-49e9-8ab9-f39f7a004443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:56.102857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d06bcc-c47f-49e9-8ab9-f39f7a004443 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:56.121514 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 988/1930] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:49:56.125757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd7421-9675-4aea-8c36-f11059eaf719 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:56.140973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd7421-9675-4aea-8c36-f11059eaf719 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:56.154585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee1cb6-dd56-414e-bcce-ff47fa243469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:56.156122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee1cb6-dd56-414e-bcce-ff47fa243469 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:56.156522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee1cb6-dd56-414e-bcce-ff47fa243469 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:56.156805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee1cb6-dd56-414e-bcce-ff47fa243469 admin admin] PATH_INFO: `/v3/projects/008f0d0aa1d843dcbe2ae42fd0028b99/users/d37a55a9f3354c749f77abbef1b57b31/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:56.206378 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee1cb6-dd56-414e-bcce-ff47fa243469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=008f0d0aa1d843dcbe2ae42fd0028b99, user_id=d37a55a9f3354c749f77abbef1b57b31, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:56.209572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee1cb6-dd56-414e-bcce-ff47fa243469 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:56.238427 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 943/1931] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:56 2022] PUT /identity/v3/projects/008f0d0aa1d843dcbe2ae42fd0028b99/users/d37a55a9f3354c749f77abbef1b57b31/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:56.250582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b8680-0ad2-41f2-a54f-e3237ecfd9fd None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:56.251550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b8680-0ad2-41f2-a54f-e3237ecfd9fd None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:56.251712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b8680-0ad2-41f2-a54f-e3237ecfd9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:56.325576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:56.355483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0d1b8680-0ad2-41f2-a54f-e3237ecfd9fd None None] MFA Rules not processed for user `d37a55a9f3354c749f77abbef1b57b31`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:56.361330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0d1b8680-0ad2-41f2-a54f-e3237ecfd9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:56.467659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 989/1932] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:56.481734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee1cb6-dd56-414e-bcce-ff47fa243469 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:56.483934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-52ee1cb6-dd56-414e-bcce-ff47fa243469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:56.545144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:56.550358 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee1cb6-dd56-414e-bcce-ff47fa243469 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:56.557832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b300d9-24d4-4180-8f85-093b4991a2ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:56.558928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b300d9-24d4-4180-8f85-093b4991a2ee service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:56.559477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b300d9-24d4-4180-8f85-093b4991a2ee service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:56.559981 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b300d9-24d4-4180-8f85-093b4991a2ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:56.572809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b300d9-24d4-4180-8f85-093b4991a2ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:56.575215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b300d9-24d4-4180-8f85-093b4991a2ee service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:56.589989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 944/1933] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:56.652669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b8680-0ad2-41f2-a54f-e3237ecfd9fd None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:56.654884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0d1b8680-0ad2-41f2-a54f-e3237ecfd9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:56.749403 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b8680-0ad2-41f2-a54f-e3237ecfd9fd None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:56.754732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:56.755616 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:56.755903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:56.756185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] PATH_INFO: `/v3/projects/dea56860adb34241bed55cef10d337dd` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:56.757439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=dea56860adb34241bed55cef10d337dd)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:56.759383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:56.815797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dea56860adb34241bed55cef10d337dd', 'request_id': 'req-4bd79659-4793-478b-9f2c-453ad4ba21fe'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:56.823297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dea56860adb34241bed55cef10d337dd', 'request_id': 'req-4bd79659-4793-478b-9f2c-453ad4ba21fe'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:56.829919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dea56860adb34241bed55cef10d337dd', 'request_id': 'req-4bd79659-4793-478b-9f2c-453ad4ba21fe'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:56.848835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] The token cache is being invalidate because project dea56860adb34241bed55cef10d337dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:56.849218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:56.849954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:56.855920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 990/1934] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:56 2022] DELETE /identity/v3/projects/dea56860adb34241bed55cef10d337dd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:56.859784 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b300d9-24d4-4180-8f85-093b4991a2ee service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:56.882975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b300d9-24d4-4180-8f85-093b4991a2ee service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:56.991202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0e9a4-43b9-46bc-a80b-8fa45f9e8f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:56.993259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0e9a4-43b9-46bc-a80b-8fa45f9e8f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:56.996509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0e9a4-43b9-46bc-a80b-8fa45f9e8f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:56.996837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0e9a4-43b9-46bc-a80b-8fa45f9e8f61 admin admin] PATH_INFO: `/v3/users/52f1df6a3a6c43729ccd561453a55b15` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:56.999993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0e9a4-43b9-46bc-a80b-8fa45f9e8f61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52f1df6a3a6c43729ccd561453a55b15)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:57.002329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0e9a4-43b9-46bc-a80b-8fa45f9e8f61 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:57.093800 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a2d0e9a4-43b9-46bc-a80b-8fa45f9e8f61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52f1df6a3a6c43729ccd561453a55b15'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.109631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a2d0e9a4-43b9-46bc-a80b-8fa45f9e8f61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52f1df6a3a6c43729ccd561453a55b15'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.119884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:57.121020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a2d0e9a4-43b9-46bc-a80b-8fa45f9e8f61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52f1df6a3a6c43729ccd561453a55b15'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.121485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:57.135506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd79659-4793-478b-9f2c-453ad4ba21fe admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:57.137672 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a2d0e9a4-43b9-46bc-a80b-8fa45f9e8f61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52f1df6a3a6c43729ccd561453a55b15'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.138212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a2d0e9a4-43b9-46bc-a80b-8fa45f9e8f61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52f1df6a3a6c43729ccd561453a55b15'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.140833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:57.143009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:57.143275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:57.143669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] PATH_INFO: `/v3/users/15ecd34476e14193b77d7341dd907578` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:57.145023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ecd34476e14193b77d7341dd907578)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:57.146315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 945/1935] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:56 2022] DELETE /identity/v3/users/52f1df6a3a6c43729ccd561453a55b15 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:57.147446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:57.177935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:57.224499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ecd34476e14193b77d7341dd907578'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.236420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ecd34476e14193b77d7341dd907578'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.245016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ecd34476e14193b77d7341dd907578'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.245634 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ecd34476e14193b77d7341dd907578'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.253782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ecd34476e14193b77d7341dd907578'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.260370 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 991/1936] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:57 2022] DELETE /identity/v3/users/15ecd34476e14193b77d7341dd907578 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:57.265024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c684b-5995-4f42-b5c5-fb81212a2757 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:57.265349 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c684b-5995-4f42-b5c5-fb81212a2757 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:57.265662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c684b-5995-4f42-b5c5-fb81212a2757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:57.284416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-e03c684b-5995-4f42-b5c5-fb81212a2757 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:57.286102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e03c684b-5995-4f42-b5c5-fb81212a2757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:57.406736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 946/1937] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:57.421929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:57.423414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:57.515428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5940fb3-7473-4c7a-9a43-019746b43db3 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:57.521049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a267c6-353f-456a-95e9-adeee0d0c817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:57.522668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a267c6-353f-456a-95e9-adeee0d0c817 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:57.523820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a267c6-353f-456a-95e9-adeee0d0c817 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:57.524213 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a267c6-353f-456a-95e9-adeee0d0c817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:57.528082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c684b-5995-4f42-b5c5-fb81212a2757 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:57.530065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e03c684b-5995-4f42-b5c5-fb81212a2757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:57.558331 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a267c6-353f-456a-95e9-adeee0d0c817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:57.559785 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a267c6-353f-456a-95e9-adeee0d0c817 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:57.565683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c684b-5995-4f42-b5c5-fb81212a2757 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:57.573401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 992/1938] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:57.578933 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:57.579829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:57.580118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:57.580396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] PATH_INFO: `/v3/projects/0a97c90093e9456aa75132cbc3c85902` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:57.581648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a97c90093e9456aa75132cbc3c85902)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:57.583588 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:57.590720 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:57.647681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a97c90093e9456aa75132cbc3c85902', 'request_id': 'req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.655712 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a97c90093e9456aa75132cbc3c85902', 'request_id': 'req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.663374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a97c90093e9456aa75132cbc3c85902', 'request_id': 'req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.686645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] The token cache is being invalidate because project 0a97c90093e9456aa75132cbc3c85902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:57.686950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.687811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7a9b8d6d-fcc3-4d50-9aca-492e3b901462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:57.694814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 947/1939] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:57 2022] DELETE /identity/v3/projects/0a97c90093e9456aa75132cbc3c85902 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:57.697570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7413-5e7f-4aef-9505-324edc3ee444 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:57.697928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7413-5e7f-4aef-9505-324edc3ee444 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:57.698735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7413-5e7f-4aef-9505-324edc3ee444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:57.726633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-c15d7413-5e7f-4aef-9505-324edc3ee444 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:57.728956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c15d7413-5e7f-4aef-9505-324edc3ee444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:57.852639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 993/1940] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:57.861528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6cc90-61a7-4535-a517-a9ed4b3e9f6a None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:57.861825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6cc90-61a7-4535-a517-a9ed4b3e9f6a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:57.862103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6cc90-61a7-4535-a517-a9ed4b3e9f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:57.888439 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-88d6cc90-61a7-4535-a517-a9ed4b3e9f6a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:57.889758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-88d6cc90-61a7-4535-a517-a9ed4b3e9f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:57.905326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 948/1941] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:57.906609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d7413-5e7f-4aef-9505-324edc3ee444 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:57.918574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d7413-5e7f-4aef-9505-324edc3ee444 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:57.923770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c583d0-8b13-40c3-8cc1-97608952f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:57.924508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c583d0-8b13-40c3-8cc1-97608952f34b admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:57.924790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c583d0-8b13-40c3-8cc1-97608952f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:57.925067 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c583d0-8b13-40c3-8cc1-97608952f34b admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:57.925836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c583d0-8b13-40c3-8cc1-97608952f34b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:57.926150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c583d0-8b13-40c3-8cc1-97608952f34b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:57.927764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c583d0-8b13-40c3-8cc1-97608952f34b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:57.943088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 994/1942] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:57.944214 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6cc90-61a7-4535-a517-a9ed4b3e9f6a None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:57.954098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6cc90-61a7-4535-a517-a9ed4b3e9f6a None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:57.964100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f65e42-1a97-420d-91e1-49f4f8c345fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:57.964988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65e42-1a97-420d-91e1-49f4f8c345fa admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:57.966771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65e42-1a97-420d-91e1-49f4f8c345fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:57.967215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65e42-1a97-420d-91e1-49f4f8c345fa admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:57.968353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f65e42-1a97-420d-91e1-49f4f8c345fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:57.971167 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f65e42-1a97-420d-91e1-49f4f8c345fa admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.007856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 949/1943] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:57 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:58.011961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c583d0-8b13-40c3-8cc1-97608952f34b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:58.022466 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c583d0-8b13-40c3-8cc1-97608952f34b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:58.058676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:58.104560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5d30d-230c-4da0-9e5f-13ffa20e8661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:58.105343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5d30d-230c-4da0-9e5f-13ffa20e8661 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.105749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5d30d-230c-4da0-9e5f-13ffa20e8661 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.106347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5d30d-230c-4da0-9e5f-13ffa20e8661 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.106886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5d30d-230c-4da0-9e5f-13ffa20e8661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:58.108984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5d30d-230c-4da0-9e5f-13ffa20e8661 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.134523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f65e42-1a97-420d-91e1-49f4f8c345fa admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:58.136377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 995/1944] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:58 2022] POST /identity/v3/users => generated 283 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:58.137162 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5d30d-230c-4da0-9e5f-13ffa20e8661 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:58.137376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-68f65e42-1a97-420d-91e1-49f4f8c345fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:58.149447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5d30d-230c-4da0-9e5f-13ffa20e8661 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:58.155475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5da4a5-0ee3-4e46-9d09-774ed97eaebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:58.156714 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5da4a5-0ee3-4e46-9d09-774ed97eaebb admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.157721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5da4a5-0ee3-4e46-9d09-774ed97eaebb admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.158058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5da4a5-0ee3-4e46-9d09-774ed97eaebb admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.158916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5da4a5-0ee3-4e46-9d09-774ed97eaebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:58.160907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5da4a5-0ee3-4e46-9d09-774ed97eaebb admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.166145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f65e42-1a97-420d-91e1-49f4f8c345fa admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:58.172650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 996/1945] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:58.174322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:58.177400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5da4a5-0ee3-4e46-9d09-774ed97eaebb admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:58.186063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.186484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.187137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] PATH_INFO: `/v3/projects/5fc379c2164c467ebc4379166ac862d3` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.188566 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fc379c2164c467ebc4379166ac862d3)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:58.189130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5da4a5-0ee3-4e46-9d09-774ed97eaebb admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:58.190536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.194676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069913a-1ee7-49c4-b1e2-61e12cf51df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:58.195416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069913a-1ee7-49c4-b1e2-61e12cf51df9 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.195772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069913a-1ee7-49c4-b1e2-61e12cf51df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.196145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069913a-1ee7-49c4-b1e2-61e12cf51df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.196977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069913a-1ee7-49c4-b1e2-61e12cf51df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:58.198798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069913a-1ee7-49c4-b1e2-61e12cf51df9 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.212008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 997/1946] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:58.217517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069913a-1ee7-49c4-b1e2-61e12cf51df9 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:58.229340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069913a-1ee7-49c4-b1e2-61e12cf51df9 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:58.236055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfc55e-238a-44a0-a80d-2056253405c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:58.237310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfc55e-238a-44a0-a80d-2056253405c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.237716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfc55e-238a-44a0-a80d-2056253405c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.238072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfc55e-238a-44a0-a80d-2056253405c8 admin admin] PATH_INFO: `/v3/projects/30e5d0b3f2f4406797a20e38cd04fd95/users/d010539016a0468984b9db6a6eb19b6c/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.258262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfc55e-238a-44a0-a80d-2056253405c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30e5d0b3f2f4406797a20e38cd04fd95, user_id=d010539016a0468984b9db6a6eb19b6c, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:58.262294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfc55e-238a-44a0-a80d-2056253405c8 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.269899 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fc379c2164c467ebc4379166ac862d3', 'request_id': 'req-c6db90dc-36e2-4d62-b557-465ddd5d00d5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:58.278763 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 998/1947] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:58 2022] PUT /identity/v3/projects/30e5d0b3f2f4406797a20e38cd04fd95/users/d010539016a0468984b9db6a6eb19b6c/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:58.295833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fc379c2164c467ebc4379166ac862d3', 'request_id': 'req-c6db90dc-36e2-4d62-b557-465ddd5d00d5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:58.297299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4bac6-555d-4ffc-843d-8f99f0a526d3 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.298363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4bac6-555d-4ffc-843d-8f99f0a526d3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.298817 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4bac6-555d-4ffc-843d-8f99f0a526d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.304159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fc379c2164c467ebc4379166ac862d3', 'request_id': 'req-c6db90dc-36e2-4d62-b557-465ddd5d00d5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:58.323310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] The token cache is being invalidate because project 5fc379c2164c467ebc4379166ac862d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:58.323656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:58.324497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:58.326502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 950/1948] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:58 2022] DELETE /identity/v3/projects/5fc379c2164c467ebc4379166ac862d3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:58.330691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:58.332177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:58.336498 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-64a4bac6-555d-4ffc-843d-8f99f0a526d3 None None] MFA Rules not processed for user `d010539016a0468984b9db6a6eb19b6c`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:58.338287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-64a4bac6-555d-4ffc-843d-8f99f0a526d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:58.393946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:58.423737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db90dc-36e2-4d62-b557-465ddd5d00d5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:58.432511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 999/1949] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:58.437128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4bac6-555d-4ffc-843d-8f99f0a526d3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:58.439929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-64a4bac6-555d-4ffc-843d-8f99f0a526d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:58.502620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:58.504047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.504503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.504863 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] PATH_INFO: `/v3/users/34d7839ef5f64fb9b09b82ded4596a2b` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.506346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34d7839ef5f64fb9b09b82ded4596a2b)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:58.508830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.515005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4bac6-555d-4ffc-843d-8f99f0a526d3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:58.520936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ecbc5-8aae-4d83-8ba7-beb262c7428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:58.521784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ecbc5-8aae-4d83-8ba7-beb262c7428f admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.522212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ecbc5-8aae-4d83-8ba7-beb262c7428f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.523044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ecbc5-8aae-4d83-8ba7-beb262c7428f admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.523350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ecbc5-8aae-4d83-8ba7-beb262c7428f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:58.525802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ecbc5-8aae-4d83-8ba7-beb262c7428f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.537409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:58.558153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1000/1950] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:58 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:58.562182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ecbc5-8aae-4d83-8ba7-beb262c7428f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:58.570250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34d7839ef5f64fb9b09b82ded4596a2b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:58.581093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34d7839ef5f64fb9b09b82ded4596a2b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:58.591790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34d7839ef5f64fb9b09b82ded4596a2b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:58.604794 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34d7839ef5f64fb9b09b82ded4596a2b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:58.605579 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34d7839ef5f64fb9b09b82ded4596a2b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:58.613320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 951/1951] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:58 2022] DELETE /identity/v3/users/34d7839ef5f64fb9b09b82ded4596a2b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:58.634258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ecbc5-8aae-4d83-8ba7-beb262c7428f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:58.716441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d3c2c-6707-45e8-829b-d2ef98715522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:58.717127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d3c2c-6707-45e8-829b-d2ef98715522 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.717447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d3c2c-6707-45e8-829b-d2ef98715522 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.718078 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d3c2c-6707-45e8-829b-d2ef98715522 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.718740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d3c2c-6707-45e8-829b-d2ef98715522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:58.720695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d3c2c-6707-45e8-829b-d2ef98715522 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.745482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1001/1952] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:58 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:49:58.747236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:58.748934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:58.769009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cbaa5-c7b1-4f2c-a943-71fe69e8caa4 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:58.774265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa32ae3-3c76-48e4-b462-c922775b0843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:58.774951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa32ae3-3c76-48e4-b462-c922775b0843 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.775235 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa32ae3-3c76-48e4-b462-c922775b0843 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.775519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa32ae3-3c76-48e4-b462-c922775b0843 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.776266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa32ae3-3c76-48e4-b462-c922775b0843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:58.777931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa32ae3-3c76-48e4-b462-c922775b0843 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.789462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 952/1953] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:58.793802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d3c2c-6707-45e8-829b-d2ef98715522 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:58.804346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d3c2c-6707-45e8-829b-d2ef98715522 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:58.809037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded10b95-3031-4859-b724-b34d67013da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:58.810210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded10b95-3031-4859-b724-b34d67013da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.810498 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded10b95-3031-4859-b724-b34d67013da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.810781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded10b95-3031-4859-b724-b34d67013da6 admin admin] PATH_INFO: `/v3/projects/df5282f89e4f4ce08f9b7f6e15ca08cd/users/a6533a3a2e524b9fa022deb684c864f4/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.833717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded10b95-3031-4859-b724-b34d67013da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5282f89e4f4ce08f9b7f6e15ca08cd, user_id=a6533a3a2e524b9fa022deb684c864f4, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:58.837048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded10b95-3031-4859-b724-b34d67013da6 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.855412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1002/1954] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:58 2022] PUT /identity/v3/projects/df5282f89e4f4ce08f9b7f6e15ca08cd/users/a6533a3a2e524b9fa022deb684c864f4/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:58.866294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa32ae3-3c76-48e4-b462-c922775b0843 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:58.882231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa32ae3-3c76-48e4-b462-c922775b0843 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:58.963631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba392f1-aa78-426c-b408-43a10c564f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:58.964634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba392f1-aa78-426c-b408-43a10c564f40 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:58.965080 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba392f1-aa78-426c-b408-43a10c564f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:58.965391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba392f1-aa78-426c-b408-43a10c564f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:58.966158 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba392f1-aa78-426c-b408-43a10c564f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:58.969064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba392f1-aa78-426c-b408-43a10c564f40 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:58.981705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:58.991248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 953/1955] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:49:58.996276 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded10b95-3031-4859-b724-b34d67013da6 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:59.009393 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded10b95-3031-4859-b724-b34d67013da6 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:59.018688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1b35b-7e2c-44a9-85dd-26eb7310ff56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:59.018688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1b35b-7e2c-44a9-85dd-26eb7310ff56 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.018688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1b35b-7e2c-44a9-85dd-26eb7310ff56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.018688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1b35b-7e2c-44a9-85dd-26eb7310ff56 admin admin] PATH_INFO: `/v3/projects/df5282f89e4f4ce08f9b7f6e15ca08cd/users/a6533a3a2e524b9fa022deb684c864f4/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.020611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1b35b-7e2c-44a9-85dd-26eb7310ff56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5282f89e4f4ce08f9b7f6e15ca08cd, user_id=a6533a3a2e524b9fa022deb684c864f4, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:59.023417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1b35b-7e2c-44a9-85dd-26eb7310ff56 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:59.041965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1003/1956] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:49:58 2022] PUT /identity/v3/projects/df5282f89e4f4ce08f9b7f6e15ca08cd/users/a6533a3a2e524b9fa022deb684c864f4/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:59.050655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0b8af-7124-4995-be13-6c262f9800c6 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.051060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0b8af-7124-4995-be13-6c262f9800c6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.051232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0b8af-7124-4995-be13-6c262f9800c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.088282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c9a0b8af-7124-4995-be13-6c262f9800c6 None None] MFA Rules not processed for user `a6533a3a2e524b9fa022deb684c864f4`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:59.089762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c9a0b8af-7124-4995-be13-6c262f9800c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:59.198850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 954/1957] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:59.211348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1b35b-7e2c-44a9-85dd-26eb7310ff56 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:59.213150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e4f1b35b-7e2c-44a9-85dd-26eb7310ff56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:59.261942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0b8af-7124-4995-be13-6c262f9800c6 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:59.265202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c9a0b8af-7124-4995-be13-6c262f9800c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:59.274304 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:59.279478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1b35b-7e2c-44a9-85dd-26eb7310ff56 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:59.285635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ebb73-eb88-4129-9eb0-c5c4a2c7e7a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:59.286391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ebb73-eb88-4129-9eb0-c5c4a2c7e7a5 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.287021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ebb73-eb88-4129-9eb0-c5c4a2c7e7a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.287367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ebb73-eb88-4129-9eb0-c5c4a2c7e7a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.301118 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ebb73-eb88-4129-9eb0-c5c4a2c7e7a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:59.303161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ebb73-eb88-4129-9eb0-c5c4a2c7e7a5 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:59.327342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1004/1958] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:59.389533 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0b8af-7124-4995-be13-6c262f9800c6 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:59.396436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:59.398077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.398850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.399352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] PATH_INFO: `/v3/projects/8bdec1ddbee7439fbaff85fc3c0d136c` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.401082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bdec1ddbee7439fbaff85fc3c0d136c)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:59.404408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:59.458324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ebb73-eb88-4129-9eb0-c5c4a2c7e7a5 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:59.461503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9f4ebb73-eb88-4129-9eb0-c5c4a2c7e7a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:59.477462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ebb73-eb88-4129-9eb0-c5c4a2c7e7a5 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:59.486320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bdec1ddbee7439fbaff85fc3c0d136c', 'request_id': 'req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:59.494934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bdec1ddbee7439fbaff85fc3c0d136c', 'request_id': 'req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:59.504099 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bdec1ddbee7439fbaff85fc3c0d136c', 'request_id': 'req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:59.520027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] The token cache is being invalidate because project 8bdec1ddbee7439fbaff85fc3c0d136c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:49:59.521175 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:59.522213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:59.528334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 955/1959] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:49:59 2022] DELETE /identity/v3/projects/8bdec1ddbee7439fbaff85fc3c0d136c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:49:59.528334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:59.528334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:59.547000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78925ecf-1efd-45f2-8960-e0d3283e8531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:59.549553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78925ecf-1efd-45f2-8960-e0d3283e8531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.550161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78925ecf-1efd-45f2-8960-e0d3283e8531 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.550278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:59.550380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78925ecf-1efd-45f2-8960-e0d3283e8531 admin admin] PATH_INFO: `/v3/users/d37a55a9f3354c749f77abbef1b57b31` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.551556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78925ecf-1efd-45f2-8960-e0d3283e8531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d37a55a9f3354c749f77abbef1b57b31)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:59.553562 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78925ecf-1efd-45f2-8960-e0d3283e8531 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:59.596747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f146a4-62ed-4925-94d6-c9e0ab8abe41 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:59.602285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d0e7c-eee0-44a8-8b5e-ffbb47154749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:59.603299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0e7c-eee0-44a8-8b5e-ffbb47154749 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.603634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0e7c-eee0-44a8-8b5e-ffbb47154749 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.604023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0e7c-eee0-44a8-8b5e-ffbb47154749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.606484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-708d0e7c-eee0-44a8-8b5e-ffbb47154749 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:59.610258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-78925ecf-1efd-45f2-8960-e0d3283e8531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd37a55a9f3354c749f77abbef1b57b31'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:59.629014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-78925ecf-1efd-45f2-8960-e0d3283e8531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd37a55a9f3354c749f77abbef1b57b31'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:59.638652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-78925ecf-1efd-45f2-8960-e0d3283e8531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd37a55a9f3354c749f77abbef1b57b31'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:59.639470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-78925ecf-1efd-45f2-8960-e0d3283e8531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd37a55a9f3354c749f77abbef1b57b31'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:59.649712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-78925ecf-1efd-45f2-8960-e0d3283e8531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd37a55a9f3354c749f77abbef1b57b31'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:49:59.655807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1005/1960] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:49:59 2022] DELETE /identity/v3/users/d37a55a9f3354c749f77abbef1b57b31 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:49:59.661943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19257ce-84cf-4f5e-befb-1256b62defce None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.662232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19257ce-84cf-4f5e-befb-1256b62defce None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.662508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19257ce-84cf-4f5e-befb-1256b62defce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.681306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d0e7c-eee0-44a8-8b5e-ffbb47154749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:59.682783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d0e7c-eee0-44a8-8b5e-ffbb47154749 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:59.683284 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e19257ce-84cf-4f5e-befb-1256b62defce None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:59.684511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e19257ce-84cf-4f5e-befb-1256b62defce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:59.692939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 956/1961] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:49:59.694750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a59426-1229-43d2-889b-ac98857d3b06 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.695048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a59426-1229-43d2-889b-ac98857d3b06 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.695335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a59426-1229-43d2-889b-ac98857d3b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.714092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-10a59426-1229-43d2-889b-ac98857d3b06 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:59.715463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-10a59426-1229-43d2-889b-ac98857d3b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:59.763219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1006/1962] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:59.775081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ace48-eec6-4f4b-b0f6-4881df53ab7b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.775458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ace48-eec6-4f4b-b0f6-4881df53ab7b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.776104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ace48-eec6-4f4b-b0f6-4881df53ab7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.796192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-9d2ace48-eec6-4f4b-b0f6-4881df53ab7b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:49:59.797428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9d2ace48-eec6-4f4b-b0f6-4881df53ab7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:59.804230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 957/1963] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:59.805742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1007/1964] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:49:59.810124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a59426-1229-43d2-889b-ac98857d3b06 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:59.810933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ace48-eec6-4f4b-b0f6-4881df53ab7b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:59.812368 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9d2ace48-eec6-4f4b-b0f6-4881df53ab7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:49:59.820125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:59.822769 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a59426-1229-43d2-889b-ac98857d3b06 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:59.831780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b12951-b885-48c8-a8ac-7776c669a4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:59.832842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b12951-b885-48c8-a8ac-7776c669a4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.832970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b12951-b885-48c8-a8ac-7776c669a4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.833402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b12951-b885-48c8-a8ac-7776c669a4a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.834342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b12951-b885-48c8-a8ac-7776c669a4a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:49:59.834638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b12951-b885-48c8-a8ac-7776c669a4a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:59.836432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b12951-b885-48c8-a8ac-7776c669a4a4 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:59.853961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 958/1965] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:49:59.858597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b12951-b885-48c8-a8ac-7776c669a4a4 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:59.871319 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b12951-b885-48c8-a8ac-7776c669a4a4 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:59.876430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2c865-99e3-49aa-bda4-2657818d84a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:59.877093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2c865-99e3-49aa-bda4-2657818d84a4 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.878406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2c865-99e3-49aa-bda4-2657818d84a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.878406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2c865-99e3-49aa-bda4-2657818d84a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.878406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2c865-99e3-49aa-bda4-2657818d84a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:59.880234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2c865-99e3-49aa-bda4-2657818d84a4 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:59.890123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ace48-eec6-4f4b-b0f6-4881df53ab7b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:59.895218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc1ef8-bc95-4dfe-812e-27f4a93e85c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:59.895889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc1ef8-bc95-4dfe-812e-27f4a93e85c6 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.896158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc1ef8-bc95-4dfe-812e-27f4a93e85c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.896472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc1ef8-bc95-4dfe-812e-27f4a93e85c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.908608 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc1ef8-bc95-4dfe-812e-27f4a93e85c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:59.909529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 959/1966] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:49:59 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:49:59.910138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc1ef8-bc95-4dfe-812e-27f4a93e85c6 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:49:59.914390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2c865-99e3-49aa-bda4-2657818d84a4 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:49:59.920436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1008/1967] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:49:59.956431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:49:59.985171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2c865-99e3-49aa-bda4-2657818d84a4 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:49:59.991294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a1ccb-0bd3-4e4b-9c60-7f182c1768a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:49:59.991979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a1ccb-0bd3-4e4b-9c60-7f182c1768a5 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:49:59.992269 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a1ccb-0bd3-4e4b-9c60-7f182c1768a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:49:59.992543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a1ccb-0bd3-4e4b-9c60-7f182c1768a5 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:49:59.993276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a1ccb-0bd3-4e4b-9c60-7f182c1768a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:49:59.995591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a1ccb-0bd3-4e4b-9c60-7f182c1768a5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:00.017245 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 960/1968] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:49:59 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:50:00.021505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc1ef8-bc95-4dfe-812e-27f4a93e85c6 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:00.033909 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc1ef8-bc95-4dfe-812e-27f4a93e85c6 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:00.038846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b50ffb-1d3b-40fc-9642-388792072c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:00.039521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b50ffb-1d3b-40fc-9642-388792072c97 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:00.040364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b50ffb-1d3b-40fc-9642-388792072c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:00.040649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b50ffb-1d3b-40fc-9642-388792072c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:00.041598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b50ffb-1d3b-40fc-9642-388792072c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:00.043174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b50ffb-1d3b-40fc-9642-388792072c97 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:00.057398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1009/1969] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:00.059078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a1ccb-0bd3-4e4b-9c60-7f182c1768a5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:00.073505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a1ccb-0bd3-4e4b-9c60-7f182c1768a5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:00.078735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6464652-4e74-4ec5-880d-98eab07643b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:00.080184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6464652-4e74-4ec5-880d-98eab07643b4 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:00.080454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6464652-4e74-4ec5-880d-98eab07643b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:00.080742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6464652-4e74-4ec5-880d-98eab07643b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:00.081521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6464652-4e74-4ec5-880d-98eab07643b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:00.083145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6464652-4e74-4ec5-880d-98eab07643b4 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:00.097113 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 961/1970] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:50:00.098338 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b50ffb-1d3b-40fc-9642-388792072c97 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:00.109539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b50ffb-1d3b-40fc-9642-388792072c97 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:00.114728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c0e69-4754-42d3-a5a2-bf9dff2f26a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:00.115676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c0e69-4754-42d3-a5a2-bf9dff2f26a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:00.115959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c0e69-4754-42d3-a5a2-bf9dff2f26a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:00.116230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c0e69-4754-42d3-a5a2-bf9dff2f26a6 admin admin] PATH_INFO: `/v3/projects/07675a13325f4517a7ffbf6a80d5a6e9/users/feb4e287f3dc4098a04f3c76c22b5667/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:00.132638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c0e69-4754-42d3-a5a2-bf9dff2f26a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07675a13325f4517a7ffbf6a80d5a6e9, user_id=feb4e287f3dc4098a04f3c76c22b5667, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:00.135349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c0e69-4754-42d3-a5a2-bf9dff2f26a6 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:00.158542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1010/1971] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:00 2022] PUT /identity/v3/projects/07675a13325f4517a7ffbf6a80d5a6e9/users/feb4e287f3dc4098a04f3c76c22b5667/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:00.169226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc5ae3-3602-4c6c-99c6-ce5a5d1d02fa None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:00.169550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc5ae3-3602-4c6c-99c6-ce5a5d1d02fa None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:00.169837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc5ae3-3602-4c6c-99c6-ce5a5d1d02fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:00.200772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a6bc5ae3-3602-4c6c-99c6-ce5a5d1d02fa None None] MFA Rules not processed for user `feb4e287f3dc4098a04f3c76c22b5667`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:00.202363 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a6bc5ae3-3602-4c6c-99c6-ce5a5d1d02fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:00.253576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c0e69-4754-42d3-a5a2-bf9dff2f26a6 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:00.255401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-789c0e69-4754-42d3-a5a2-bf9dff2f26a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:00.285428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 962/1972] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:00.298650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc5ae3-3602-4c6c-99c6-ce5a5d1d02fa None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:00.300530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a6bc5ae3-3602-4c6c-99c6-ce5a5d1d02fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:00.332591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c0e69-4754-42d3-a5a2-bf9dff2f26a6 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:00.337675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:00.338553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:00.338836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:00.339113 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] PATH_INFO: `/v3/projects/008f0d0aa1d843dcbe2ae42fd0028b99` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:00.340490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=008f0d0aa1d843dcbe2ae42fd0028b99)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:00.342786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:00.399981 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc5ae3-3602-4c6c-99c6-ce5a5d1d02fa None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:00.407652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '008f0d0aa1d843dcbe2ae42fd0028b99', 'request_id': 'req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:00.418237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '008f0d0aa1d843dcbe2ae42fd0028b99', 'request_id': 'req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:00.427708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '008f0d0aa1d843dcbe2ae42fd0028b99', 'request_id': 'req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:00.440099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] The token cache is being invalidate because project 008f0d0aa1d843dcbe2ae42fd0028b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:00.440524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:00.441476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e621aa45-8ef1-4f8c-9dfc-fc6c0b391b57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:00.447068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1011/1973] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:00 2022] DELETE /identity/v3/projects/008f0d0aa1d843dcbe2ae42fd0028b99 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:00.448951 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd64563-0bc5-4382-b365-d43d50eb8f50 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:00.449401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd64563-0bc5-4382-b365-d43d50eb8f50 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:00.450014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd64563-0bc5-4382-b365-d43d50eb8f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:00.459832 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:00.469779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968eaf8-d3e9-4cdf-bb50-cde7ab57d322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:00.471421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968eaf8-d3e9-4cdf-bb50-cde7ab57d322 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:00.471679 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968eaf8-d3e9-4cdf-bb50-cde7ab57d322 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:00.472000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968eaf8-d3e9-4cdf-bb50-cde7ab57d322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:00.474177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-2cd64563-0bc5-4382-b365-d43d50eb8f50 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:00.474644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7968eaf8-d3e9-4cdf-bb50-cde7ab57d322 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:00.476468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2cd64563-0bc5-4382-b365-d43d50eb8f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:00.498424 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:00.637014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968eaf8-d3e9-4cdf-bb50-cde7ab57d322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:00.638452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968eaf8-d3e9-4cdf-bb50-cde7ab57d322 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:00.669532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1012/1974] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:00.677618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed610cd-ff6f-4628-9381-cdf1f08e2bd6 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:00.677916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed610cd-ff6f-4628-9381-cdf1f08e2bd6 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:00.678205 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed610cd-ff6f-4628-9381-cdf1f08e2bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:00.683098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 963/1975] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:00.716126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-3ed610cd-ff6f-4628-9381-cdf1f08e2bd6 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:00.717284 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3ed610cd-ff6f-4628-9381-cdf1f08e2bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:00.725490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968eaf8-d3e9-4cdf-bb50-cde7ab57d322 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:00.727195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7968eaf8-d3e9-4cdf-bb50-cde7ab57d322 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:00.736387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1013/1976] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:00.737387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed610cd-ff6f-4628-9381-cdf1f08e2bd6 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:00.756980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed610cd-ff6f-4628-9381-cdf1f08e2bd6 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:00.758838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968eaf8-d3e9-4cdf-bb50-cde7ab57d322 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:00.764511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4aee0d-ed73-4bdf-b382-bcc8ac55e956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:00.765317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aee0d-ed73-4bdf-b382-bcc8ac55e956 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:00.765778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aee0d-ed73-4bdf-b382-bcc8ac55e956 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:00.766118 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4aee0d-ed73-4bdf-b382-bcc8ac55e956 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:00.772811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aee0d-ed73-4bdf-b382-bcc8ac55e956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:00.773296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aee0d-ed73-4bdf-b382-bcc8ac55e956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:00.775044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4aee0d-ed73-4bdf-b382-bcc8ac55e956 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:00.779585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abadc315-7382-449c-8635-99a494d337ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:00.781489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadc315-7382-449c-8635-99a494d337ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:00.781779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadc315-7382-449c-8635-99a494d337ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:00.782106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abadc315-7382-449c-8635-99a494d337ea admin admin] PATH_INFO: `/v3/users/feb4e287f3dc4098a04f3c76c22b5667` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:00.787404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abadc315-7382-449c-8635-99a494d337ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb4e287f3dc4098a04f3c76c22b5667)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:00.792150 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abadc315-7382-449c-8635-99a494d337ea admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:00.803628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1014/1977] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:50:00.810020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4aee0d-ed73-4bdf-b382-bcc8ac55e956 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:00.834752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4aee0d-ed73-4bdf-b382-bcc8ac55e956 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:00.854891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9f863-e6b8-4976-8a5e-f0c429781ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:00.854891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9f863-e6b8-4976-8a5e-f0c429781ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:00.855187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9f863-e6b8-4976-8a5e-f0c429781ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:00.855531 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9f863-e6b8-4976-8a5e-f0c429781ac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:00.856276 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9f863-e6b8-4976-8a5e-f0c429781ac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:00.858935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9f863-e6b8-4976-8a5e-f0c429781ac3 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:00.916895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1015/1978] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:00 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:00.923414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9f863-e6b8-4976-8a5e-f0c429781ac3 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:00.980110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-abadc315-7382-449c-8635-99a494d337ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb4e287f3dc4098a04f3c76c22b5667'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:01.003035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-abadc315-7382-449c-8635-99a494d337ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb4e287f3dc4098a04f3c76c22b5667'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:01.023656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-abadc315-7382-449c-8635-99a494d337ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb4e287f3dc4098a04f3c76c22b5667'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:01.056314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-abadc315-7382-449c-8635-99a494d337ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb4e287f3dc4098a04f3c76c22b5667'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:01.058895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-abadc315-7382-449c-8635-99a494d337ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb4e287f3dc4098a04f3c76c22b5667'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:01.072250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 964/1979] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:00 2022] DELETE /identity/v3/users/feb4e287f3dc4098a04f3c76c22b5667 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:01.077582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b20b9a-37e5-4341-b0a6-ac36fe20d3d1 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:01.082819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b20b9a-37e5-4341-b0a6-ac36fe20d3d1 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:01.082819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b20b9a-37e5-4341-b0a6-ac36fe20d3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:01.108864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9f863-e6b8-4976-8a5e-f0c429781ac3 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:01.125057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-b6b20b9a-37e5-4341-b0a6-ac36fe20d3d1 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:01.126385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b6b20b9a-37e5-4341-b0a6-ac36fe20d3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:01.242822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bd319-e2c4-421e-92e8-57c1f245a459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:01.243912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bd319-e2c4-421e-92e8-57c1f245a459 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:01.244434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bd319-e2c4-421e-92e8-57c1f245a459 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:01.244876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bd319-e2c4-421e-92e8-57c1f245a459 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:01.245892 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717bd319-e2c4-421e-92e8-57c1f245a459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:01.247968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717bd319-e2c4-421e-92e8-57c1f245a459 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:01.271071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 965/1980] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:01.271071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b20b9a-37e5-4341-b0a6-ac36fe20d3d1 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:01.273570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:01.279802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b6b20b9a-37e5-4341-b0a6-ac36fe20d3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:01.291293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1016/1981] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:00 2022] POST /identity/v3/users => generated 286 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:01.292321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bd319-e2c4-421e-92e8-57c1f245a459 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:01.305986 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-717bd319-e2c4-421e-92e8-57c1f245a459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:01.316567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b20b9a-37e5-4341-b0a6-ac36fe20d3d1 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:01.325758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:01.326903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:01.327461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:01.328282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] PATH_INFO: `/v3/users/f6a4b7a1b7ad4e2db44cd55f5485aca9` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:01.333631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a4b7a1b7ad4e2db44cd55f5485aca9)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:01.343751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:01.385868 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bd319-e2c4-421e-92e8-57c1f245a459 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:01.397872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbd5d4-6811-4579-be38-4c8ef3c87dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:01.399055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbd5d4-6811-4579-be38-4c8ef3c87dab service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:01.399342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbd5d4-6811-4579-be38-4c8ef3c87dab service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:01.402179 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbd5d4-6811-4579-be38-4c8ef3c87dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:01.410271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:01.423762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbd5d4-6811-4579-be38-4c8ef3c87dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:01.429425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbd5d4-6811-4579-be38-4c8ef3c87dab service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:01.443080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1017/1982] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:01.444014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbd5d4-6811-4579-be38-4c8ef3c87dab service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:01.445524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-19dbd5d4-6811-4579-be38-4c8ef3c87dab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:01.460320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a4b7a1b7ad4e2db44cd55f5485aca9'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:01.460653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbd5d4-6811-4579-be38-4c8ef3c87dab service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:01.466293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496214e-f818-4d38-939f-a665c4484cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:01.466976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496214e-f818-4d38-939f-a665c4484cfd admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:01.467274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496214e-f818-4d38-939f-a665c4484cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:01.467676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496214e-f818-4d38-939f-a665c4484cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:01.468876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496214e-f818-4d38-939f-a665c4484cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:01.471180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496214e-f818-4d38-939f-a665c4484cfd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:01.480464 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a4b7a1b7ad4e2db44cd55f5485aca9'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:01.484794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1018/1983] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:01.489693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496214e-f818-4d38-939f-a665c4484cfd admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:01.492248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a4b7a1b7ad4e2db44cd55f5485aca9'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:01.501080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496214e-f818-4d38-939f-a665c4484cfd admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:01.509796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b7d3d-8e28-497b-a709-3f162ff1f96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:01.510686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7d3d-8e28-497b-a709-3f162ff1f96b admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:01.510984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7d3d-8e28-497b-a709-3f162ff1f96b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:01.511097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a4b7a1b7ad4e2db44cd55f5485aca9'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:01.511259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b7d3d-8e28-497b-a709-3f162ff1f96b admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:01.512181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b7d3d-8e28-497b-a709-3f162ff1f96b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:01.512438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a4b7a1b7ad4e2db44cd55f5485aca9'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:01.514346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b7d3d-8e28-497b-a709-3f162ff1f96b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:01.528434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 966/1984] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:01 2022] DELETE /identity/v3/users/f6a4b7a1b7ad4e2db44cd55f5485aca9 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:01.531210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1019/1985] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:01.532838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:01.535228 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f673a2-d36b-47ea-87ec-86bae6409ac3 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:01.535627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f673a2-d36b-47ea-87ec-86bae6409ac3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:01.535884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:01.536119 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f673a2-d36b-47ea-87ec-86bae6409ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:01.557546 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a8f673a2-d36b-47ea-87ec-86bae6409ac3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:01.558920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a8f673a2-d36b-47ea-87ec-86bae6409ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:01.633822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1020/1986] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:01.639281 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fec1f9-5069-4ac3-b3c3-e339dce08661 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:01.645452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f673a2-d36b-47ea-87ec-86bae6409ac3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:01.645650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092318a-974a-483c-960e-ca6dd24cbe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:01.646775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092318a-974a-483c-960e-ca6dd24cbe50 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:01.647062 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092318a-974a-483c-960e-ca6dd24cbe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:01.647339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a8f673a2-d36b-47ea-87ec-86bae6409ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:01.647448 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092318a-974a-483c-960e-ca6dd24cbe50 admin admin] PATH_INFO: `/v3/projects/fa251f62e3ba456ba3d8a23b9f137afb/users/72b980470adc4ba0b391f70219eca8ab/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:01.665843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092318a-974a-483c-960e-ca6dd24cbe50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa251f62e3ba456ba3d8a23b9f137afb, user_id=72b980470adc4ba0b391f70219eca8ab, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:01.668514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092318a-974a-483c-960e-ca6dd24cbe50 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:01.701883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 967/1987] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:01 2022] PUT /identity/v3/projects/fa251f62e3ba456ba3d8a23b9f137afb/users/72b980470adc4ba0b391f70219eca8ab/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:01.703483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4bc72-d201-4340-b2ac-5226cba0b1f8 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:01.704457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4bc72-d201-4340-b2ac-5226cba0b1f8 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:01.704850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4bc72-d201-4340-b2ac-5226cba0b1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:01.735001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c8f4bc72-d201-4340-b2ac-5226cba0b1f8 None None] MFA Rules not processed for user `72b980470adc4ba0b391f70219eca8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:01.736379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c8f4bc72-d201-4340-b2ac-5226cba0b1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:01.742308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f673a2-d36b-47ea-87ec-86bae6409ac3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:01.798964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:01.831732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4724-52e7-4ce4-a688-680a768ae368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:01.832580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4724-52e7-4ce4-a688-680a768ae368 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:01.832965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4724-52e7-4ce4-a688-680a768ae368 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:01.833666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4724-52e7-4ce4-a688-680a768ae368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:01.841669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 968/1988] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:01.854172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4bc72-d201-4340-b2ac-5226cba0b1f8 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:01.856622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c8f4bc72-d201-4340-b2ac-5226cba0b1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:01.893585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:02.002304 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed4724-52e7-4ce4-a688-680a768ae368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:02.009649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed4724-52e7-4ce4-a688-680a768ae368 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:02.018522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4bc72-d201-4340-b2ac-5226cba0b1f8 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:02.031605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a8516-a788-4576-b6e5-a72e4399068e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82e1192b2d554199a08debb4aa392479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:02.034049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1021/1989] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:02.035556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a8516-a788-4576-b6e5-a72e4399068e service glance] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.035977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a8516-a788-4576-b6e5-a72e4399068e service glance] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.036593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a8516-a788-4576-b6e5-a72e4399068e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.048143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a8516-a788-4576-b6e5-a72e4399068e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:02.049903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a8516-a788-4576-b6e5-a72e4399068e service glance] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:02.061837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4724-52e7-4ce4-a688-680a768ae368 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:02.066837 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-89ed4724-52e7-4ce4-a688-680a768ae368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:02.079902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 969/1990] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:50:02.125661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4724-52e7-4ce4-a688-680a768ae368 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:02.135148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:02.136120 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.138223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.138519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] PATH_INFO: `/v3/projects/07675a13325f4517a7ffbf6a80d5a6e9` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.140404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07675a13325f4517a7ffbf6a80d5a6e9)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:02.142805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:02.218996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07675a13325f4517a7ffbf6a80d5a6e9', 'request_id': 'req-dba31722-7959-442f-a31f-c660e3b90375'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.230227 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07675a13325f4517a7ffbf6a80d5a6e9', 'request_id': 'req-dba31722-7959-442f-a31f-c660e3b90375'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.240325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07675a13325f4517a7ffbf6a80d5a6e9', 'request_id': 'req-dba31722-7959-442f-a31f-c660e3b90375'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.260519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] The token cache is being invalidate because project 07675a13325f4517a7ffbf6a80d5a6e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:02.261074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.262547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-dba31722-7959-442f-a31f-c660e3b90375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.265814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1022/1991] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:02 2022] DELETE /identity/v3/projects/07675a13325f4517a7ffbf6a80d5a6e9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:02.270683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fceae-7140-49e7-860c-e07d42deb45e None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.271126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fceae-7140-49e7-860c-e07d42deb45e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.271502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fceae-7140-49e7-860c-e07d42deb45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.311490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-466fceae-7140-49e7-860c-e07d42deb45e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:02.314185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-466fceae-7140-49e7-860c-e07d42deb45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:02.389832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 970/1992] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:02.399823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8aaed-8781-4bb8-b21f-f92cdada7139 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.400200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8aaed-8781-4bb8-b21f-f92cdada7139 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.400627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8aaed-8781-4bb8-b21f-f92cdada7139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.416057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fceae-7140-49e7-860c-e07d42deb45e None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:02.418153 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-466fceae-7140-49e7-860c-e07d42deb45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:02.429935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0fc8aaed-8781-4bb8-b21f-f92cdada7139 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:02.431539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0fc8aaed-8781-4bb8-b21f-f92cdada7139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:02.436831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fceae-7140-49e7-860c-e07d42deb45e None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:02.441533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1023/1993] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:02.445866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8aaed-8781-4bb8-b21f-f92cdada7139 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:02.450052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:02.450943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.451233 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.451516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] PATH_INFO: `/v3/projects/40ef448d42fb4eb786f24409f00d808c` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.452912 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ef448d42fb4eb786f24409f00d808c)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:02.456460 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:02.457923 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8aaed-8781-4bb8-b21f-f92cdada7139 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:02.462968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45f097-3acd-44fd-bf6a-e53d33a2da24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:02.463672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f097-3acd-44fd-bf6a-e53d33a2da24 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.463977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f097-3acd-44fd-bf6a-e53d33a2da24 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.464237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f097-3acd-44fd-bf6a-e53d33a2da24 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.465235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45f097-3acd-44fd-bf6a-e53d33a2da24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:02.465594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45f097-3acd-44fd-bf6a-e53d33a2da24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:02.467084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45f097-3acd-44fd-bf6a-e53d33a2da24 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:02.481132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1024/1994] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:50:02.485845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45f097-3acd-44fd-bf6a-e53d33a2da24 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:02.497006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45f097-3acd-44fd-bf6a-e53d33a2da24 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:02.503082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269c3cb-a566-4817-b873-c26d9aaa083b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:02.503774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269c3cb-a566-4817-b873-c26d9aaa083b admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.504049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269c3cb-a566-4817-b873-c26d9aaa083b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.504494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269c3cb-a566-4817-b873-c26d9aaa083b admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.505039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269c3cb-a566-4817-b873-c26d9aaa083b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:02.507082 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269c3cb-a566-4817-b873-c26d9aaa083b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:02.515540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ef448d42fb4eb786f24409f00d808c', 'request_id': 'req-6bf62069-18b6-4586-a020-555a9034ce95'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.534556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ef448d42fb4eb786f24409f00d808c', 'request_id': 'req-6bf62069-18b6-4586-a020-555a9034ce95'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.542393 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1025/1995] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:02 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:02.544147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ef448d42fb4eb786f24409f00d808c', 'request_id': 'req-6bf62069-18b6-4586-a020-555a9034ce95'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.546748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269c3cb-a566-4817-b873-c26d9aaa083b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:02.564972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] The token cache is being invalidate because project 40ef448d42fb4eb786f24409f00d808c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:02.565252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.566191 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.568673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 971/1996] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:02 2022] DELETE /identity/v3/projects/40ef448d42fb4eb786f24409f00d808c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:02.572773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:02.574914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:02.600025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:02.612321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:02.662269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269c3cb-a566-4817-b873-c26d9aaa083b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:02.667374 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a7b7a-6b29-417c-bfd4-4e13ef6b4b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:02.668173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a7b7a-6b29-417c-bfd4-4e13ef6b4b3c admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.668673 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a7b7a-6b29-417c-bfd4-4e13ef6b4b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.669073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a7b7a-6b29-417c-bfd4-4e13ef6b4b3c admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.669905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a7b7a-6b29-417c-bfd4-4e13ef6b4b3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:02.672037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a7b7a-6b29-417c-bfd4-4e13ef6b4b3c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:02.691007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf62069-18b6-4586-a020-555a9034ce95 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:02.700151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1026/1997] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:02 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:02.706226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b188fe-7128-4417-a25d-b02cb0ca6752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:02.706516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a7b7a-6b29-417c-bfd4-4e13ef6b4b3c admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:02.707185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b188fe-7128-4417-a25d-b02cb0ca6752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.707426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b188fe-7128-4417-a25d-b02cb0ca6752 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.707723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b188fe-7128-4417-a25d-b02cb0ca6752 admin admin] PATH_INFO: `/v3/users/95c8f2010d4a4e4897fa73e39b64a1ce` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.711073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-be9a7b7a-6b29-417c-bfd4-4e13ef6b4b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:02.711327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b188fe-7128-4417-a25d-b02cb0ca6752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c8f2010d4a4e4897fa73e39b64a1ce)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:02.715250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b188fe-7128-4417-a25d-b02cb0ca6752 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:02.728400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a7b7a-6b29-417c-bfd4-4e13ef6b4b3c admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:02.740438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090997a-67d6-4150-9e77-8f0cff7b4c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:02.741509 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090997a-67d6-4150-9e77-8f0cff7b4c9c admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.742079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090997a-67d6-4150-9e77-8f0cff7b4c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.742445 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090997a-67d6-4150-9e77-8f0cff7b4c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.743194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090997a-67d6-4150-9e77-8f0cff7b4c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:02.744834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090997a-67d6-4150-9e77-8f0cff7b4c9c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:02.759359 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1027/1998] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:02.773619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090997a-67d6-4150-9e77-8f0cff7b4c9c admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:02.793662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090997a-67d6-4150-9e77-8f0cff7b4c9c admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:02.800783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3a031-c871-41df-a24c-d590d25ccdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:02.801655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3a031-c871-41df-a24c-d590d25ccdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.802036 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3a031-c871-41df-a24c-d590d25ccdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.802379 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3a031-c871-41df-a24c-d590d25ccdd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.803190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3a031-c871-41df-a24c-d590d25ccdd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:02.805024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3a031-c871-41df-a24c-d590d25ccdd7 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:02.823520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1028/1999] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:02.827609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3a031-c871-41df-a24c-d590d25ccdd7 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:02.840125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3a031-c871-41df-a24c-d590d25ccdd7 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:02.840471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-70b188fe-7128-4417-a25d-b02cb0ca6752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c8f2010d4a4e4897fa73e39b64a1ce'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.848080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8e38e-4423-4a4c-a803-a1cfba7f0cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:02.849978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8e38e-4423-4a4c-a803-a1cfba7f0cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.850253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8e38e-4423-4a4c-a803-a1cfba7f0cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.850760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8e38e-4423-4a4c-a803-a1cfba7f0cb5 admin admin] PATH_INFO: `/v3/projects/6912a800467c48559b89133c81b104ba/users/111ca1623b6046549ee7e93802ab7ed2/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:02.854354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-70b188fe-7128-4417-a25d-b02cb0ca6752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c8f2010d4a4e4897fa73e39b64a1ce'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.867922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-70b188fe-7128-4417-a25d-b02cb0ca6752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c8f2010d4a4e4897fa73e39b64a1ce'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.871325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8e38e-4423-4a4c-a803-a1cfba7f0cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6912a800467c48559b89133c81b104ba, user_id=111ca1623b6046549ee7e93802ab7ed2, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:02.874475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8e38e-4423-4a4c-a803-a1cfba7f0cb5 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:02.893999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-70b188fe-7128-4417-a25d-b02cb0ca6752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c8f2010d4a4e4897fa73e39b64a1ce'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.895620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-70b188fe-7128-4417-a25d-b02cb0ca6752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c8f2010d4a4e4897fa73e39b64a1ce'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:02.904548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 972/2000] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:02 2022] DELETE /identity/v3/users/95c8f2010d4a4e4897fa73e39b64a1ce => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:02.904648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1029/2001] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:02 2022] PUT /identity/v3/projects/6912a800467c48559b89133c81b104ba/users/111ca1623b6046549ee7e93802ab7ed2/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:02.925707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16491d9e-e3e3-4d1a-ac3d-34ca90298001 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:02.927798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16491d9e-e3e3-4d1a-ac3d-34ca90298001 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:02.928431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16491d9e-e3e3-4d1a-ac3d-34ca90298001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:03.015791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-16491d9e-e3e3-4d1a-ac3d-34ca90298001 None None] MFA Rules not processed for user `111ca1623b6046549ee7e93802ab7ed2`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:03.017669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-16491d9e-e3e3-4d1a-ac3d-34ca90298001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:03.046649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8e38e-4423-4a4c-a803-a1cfba7f0cb5 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:03.051418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a0a8e38e-4423-4a4c-a803-a1cfba7f0cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:03.134751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 973/2002] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:03.141278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16491d9e-e3e3-4d1a-ac3d-34ca90298001 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:03.143337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-16491d9e-e3e3-4d1a-ac3d-34ca90298001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:03.159153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8e38e-4423-4a4c-a803-a1cfba7f0cb5 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:03.191807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:03.250652 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16491d9e-e3e3-4d1a-ac3d-34ca90298001 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:03.259766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb8b7b-01ff-4358-9da0-7c23a9479344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:03.261120 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb8b7b-01ff-4358-9da0-7c23a9479344 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:03.261605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb8b7b-01ff-4358-9da0-7c23a9479344 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:03.262492 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb8b7b-01ff-4358-9da0-7c23a9479344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:03.275167 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb8b7b-01ff-4358-9da0-7c23a9479344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:03.276980 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb8b7b-01ff-4358-9da0-7c23a9479344 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:03.278151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7694d-8782-4369-bc34-769457d0f4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:03.279039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7694d-8782-4369-bc34-769457d0f4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:03.279317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7694d-8782-4369-bc34-769457d0f4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:03.279590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7694d-8782-4369-bc34-769457d0f4ea admin admin] PATH_INFO: `/v3/users/d010539016a0468984b9db6a6eb19b6c` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:03.280978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7694d-8782-4369-bc34-769457d0f4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d010539016a0468984b9db6a6eb19b6c)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:03.283815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7694d-8782-4369-bc34-769457d0f4ea admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:03.305724 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 974/2003] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:03.335216 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:03.387348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4ff7694d-8782-4369-bc34-769457d0f4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd010539016a0468984b9db6a6eb19b6c'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:03.425980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4ff7694d-8782-4369-bc34-769457d0f4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd010539016a0468984b9db6a6eb19b6c'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:03.436848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4ff7694d-8782-4369-bc34-769457d0f4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd010539016a0468984b9db6a6eb19b6c'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:03.438484 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4ff7694d-8782-4369-bc34-769457d0f4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd010539016a0468984b9db6a6eb19b6c'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:03.460455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4ff7694d-8782-4369-bc34-769457d0f4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd010539016a0468984b9db6a6eb19b6c'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:03.469103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb8b7b-01ff-4358-9da0-7c23a9479344 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:03.471344 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-90eb8b7b-01ff-4358-9da0-7c23a9479344 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:03.474493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1030/2004] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:03 2022] DELETE /identity/v3/users/d010539016a0468984b9db6a6eb19b6c => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:03.479735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5cd6c-dc29-4bde-90a7-742ec12275b8 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:03.480476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5cd6c-dc29-4bde-90a7-742ec12275b8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:03.488093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5cd6c-dc29-4bde-90a7-742ec12275b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:03.496093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb8b7b-01ff-4358-9da0-7c23a9479344 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:03.511511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-1af5cd6c-dc29-4bde-90a7-742ec12275b8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:03.512748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1af5cd6c-dc29-4bde-90a7-742ec12275b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:03.585395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1031/2005] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:03.586974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:03.587854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:03.589461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:03.589853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] PATH_INFO: `/v3/projects/c6ab56d8b453443ca35b0eb4c6a33b9f` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:03.591662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ab56d8b453443ca35b0eb4c6a33b9f)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:03.595966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5cd6c-dc29-4bde-90a7-742ec12275b8 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:03.596770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:03.598411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1af5cd6c-dc29-4bde-90a7-742ec12275b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:03.655561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ab56d8b453443ca35b0eb4c6a33b9f', 'request_id': 'req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:03.664223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ab56d8b453443ca35b0eb4c6a33b9f', 'request_id': 'req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:03.673119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ab56d8b453443ca35b0eb4c6a33b9f', 'request_id': 'req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:03.674053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5cd6c-dc29-4bde-90a7-742ec12275b8 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:03.694398 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] The token cache is being invalidate because project c6ab56d8b453443ca35b0eb4c6a33b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:03.694923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:03.695344 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bd497ca9-664b-4688-ba2d-bb75bc7ddf2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:03.696903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 975/2006] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:03 2022] DELETE /identity/v3/projects/c6ab56d8b453443ca35b0eb4c6a33b9f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:03.701666 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f2e9a-ebcc-4de4-ad97-436902cfbd93 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:03.702015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f2e9a-ebcc-4de4-ad97-436902cfbd93 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:03.702365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f2e9a-ebcc-4de4-ad97-436902cfbd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:03.730147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-165f2e9a-ebcc-4de4-ad97-436902cfbd93 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:03.731734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-165f2e9a-ebcc-4de4-ad97-436902cfbd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:03.759611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704557a4-e114-4fb6-9f2b-906f3644ede9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:03.760291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704557a4-e114-4fb6-9f2b-906f3644ede9 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:03.760576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704557a4-e114-4fb6-9f2b-906f3644ede9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:03.760851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704557a4-e114-4fb6-9f2b-906f3644ede9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:03.762764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-704557a4-e114-4fb6-9f2b-906f3644ede9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:03.771095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:03.819122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:03.835575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 976/2007] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:03.843450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843b3af-d148-4e12-b403-9676923d026f None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:03.843765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843b3af-d148-4e12-b403-9676923d026f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:03.844123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843b3af-d148-4e12-b403-9676923d026f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:03.850150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704557a4-e114-4fb6-9f2b-906f3644ede9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:03.851599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704557a4-e114-4fb6-9f2b-906f3644ede9 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:03.861803 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1032/2008] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:50:03.862648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0843b3af-d148-4e12-b403-9676923d026f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:03.863860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0843b3af-d148-4e12-b403-9676923d026f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:03.871996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 977/2009] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:03.876769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704557a4-e114-4fb6-9f2b-906f3644ede9 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:03.886406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704557a4-e114-4fb6-9f2b-906f3644ede9 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:03.891220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bf26c-6bf3-4849-9d11-dcf1ba510ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:03.892455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bf26c-6bf3-4849-9d11-dcf1ba510ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:03.892743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bf26c-6bf3-4849-9d11-dcf1ba510ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:03.893041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bf26c-6bf3-4849-9d11-dcf1ba510ba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:03.893807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bf26c-6bf3-4849-9d11-dcf1ba510ba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:03.894118 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bf26c-6bf3-4849-9d11-dcf1ba510ba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:03.895707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bf26c-6bf3-4849-9d11-dcf1ba510ba2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:03.908014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1033/2010] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:50:03.912655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843b3af-d148-4e12-b403-9676923d026f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:03.923053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843b3af-d148-4e12-b403-9676923d026f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:03.928128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be6ab-8d7b-4836-88f8-58c339d36f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:03.929041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be6ab-8d7b-4836-88f8-58c339d36f17 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:03.929041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be6ab-8d7b-4836-88f8-58c339d36f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:03.929349 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be6ab-8d7b-4836-88f8-58c339d36f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:03.930088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92be6ab-8d7b-4836-88f8-58c339d36f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:03.931977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92be6ab-8d7b-4836-88f8-58c339d36f17 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:03.976430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bf26c-6bf3-4849-9d11-dcf1ba510ba2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:03.981569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 978/2011] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:03 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:03.990339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bf26c-6bf3-4849-9d11-dcf1ba510ba2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:04.097485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad662c-ce73-4395-88d2-fb112b86c702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:04.098710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad662c-ce73-4395-88d2-fb112b86c702 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:04.099175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad662c-ce73-4395-88d2-fb112b86c702 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:04.099706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad662c-ce73-4395-88d2-fb112b86c702 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:04.101998 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad662c-ce73-4395-88d2-fb112b86c702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:04.104157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad662c-ce73-4395-88d2-fb112b86c702 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:04.148907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1034/2012] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:03 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:04.153639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be6ab-8d7b-4836-88f8-58c339d36f17 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:04.165998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be6ab-8d7b-4836-88f8-58c339d36f17 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:04.172029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13f89b-6c2b-46d2-9d17-d2811334d9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:04.172766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f89b-6c2b-46d2-9d17-d2811334d9ea admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:04.173203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f89b-6c2b-46d2-9d17-d2811334d9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:04.173531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f89b-6c2b-46d2-9d17-d2811334d9ea admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:04.174354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13f89b-6c2b-46d2-9d17-d2811334d9ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:04.176277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13f89b-6c2b-46d2-9d17-d2811334d9ea admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:04.189405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 979/2013] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:04.193565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad662c-ce73-4395-88d2-fb112b86c702 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:04.205091 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad662c-ce73-4395-88d2-fb112b86c702 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:04.210104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d4f92-608f-4a9a-b133-9bfa956df1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:04.210785 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d4f92-608f-4a9a-b133-9bfa956df1dc admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:04.211093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d4f92-608f-4a9a-b133-9bfa956df1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:04.211372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d4f92-608f-4a9a-b133-9bfa956df1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:04.212104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d4f92-608f-4a9a-b133-9bfa956df1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:04.217294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d4f92-608f-4a9a-b133-9bfa956df1dc admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:04.232358 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1035/2014] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:04.237833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13f89b-6c2b-46d2-9d17-d2811334d9ea admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:04.253752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13f89b-6c2b-46d2-9d17-d2811334d9ea admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:04.263500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbc3db-6e78-4dd0-819a-e0c42892e27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:04.264562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbc3db-6e78-4dd0-819a-e0c42892e27f admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:04.265961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbc3db-6e78-4dd0-819a-e0c42892e27f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:04.266380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbc3db-6e78-4dd0-819a-e0c42892e27f admin admin] PATH_INFO: `/v3/projects/32b877071c79438292e23af5f0401e6c/users/8927179bd32842fb90fe6d9c7b903aca/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:04.303816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dbc3db-6e78-4dd0-819a-e0c42892e27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=32b877071c79438292e23af5f0401e6c, user_id=8927179bd32842fb90fe6d9c7b903aca, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:04.306621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dbc3db-6e78-4dd0-819a-e0c42892e27f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:04.328526 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 980/2015] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:04 2022] PUT /identity/v3/projects/32b877071c79438292e23af5f0401e6c/users/8927179bd32842fb90fe6d9c7b903aca/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:04.340647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc603bd7-8bbb-4355-bb1d-22425c54ceab None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:04.341001 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc603bd7-8bbb-4355-bb1d-22425c54ceab None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:04.341201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc603bd7-8bbb-4355-bb1d-22425c54ceab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:04.382377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-fc603bd7-8bbb-4355-bb1d-22425c54ceab None None] MFA Rules not processed for user `8927179bd32842fb90fe6d9c7b903aca`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:04.383625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fc603bd7-8bbb-4355-bb1d-22425c54ceab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:04.481144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:04.489807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1036/2016] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:04.493615 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbc3db-6e78-4dd0-819a-e0c42892e27f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:04.569789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbc3db-6e78-4dd0-819a-e0c42892e27f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:04.574690 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4d379-d56f-445c-90c3-16af1f756ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:04.575593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4d379-d56f-445c-90c3-16af1f756ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:04.575733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4d379-d56f-445c-90c3-16af1f756ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:04.575955 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4d379-d56f-445c-90c3-16af1f756ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:04.576669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4d379-d56f-445c-90c3-16af1f756ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:04.578728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4d379-d56f-445c-90c3-16af1f756ff5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:04.627878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 981/2017] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:04 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:04.631760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc603bd7-8bbb-4355-bb1d-22425c54ceab None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:04.644444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc603bd7-8bbb-4355-bb1d-22425c54ceab None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:04.738958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a06a8-072f-453c-9064-9a77212fad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:04.739917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a06a8-072f-453c-9064-9a77212fad35 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:04.740252 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a06a8-072f-453c-9064-9a77212fad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:04.740536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a06a8-072f-453c-9064-9a77212fad35 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:04.741409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a06a8-072f-453c-9064-9a77212fad35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:04.743703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a06a8-072f-453c-9064-9a77212fad35 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:04.795214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1037/2018] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:04 2022] POST /identity/v3/users => generated 300 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:04.799884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4d379-d56f-445c-90c3-16af1f756ff5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:04.819551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4d379-d56f-445c-90c3-16af1f756ff5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:04.836594 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee79eb-e8a2-4d4e-807d-338aaef197ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:04.837617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee79eb-e8a2-4d4e-807d-338aaef197ee admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:04.840303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee79eb-e8a2-4d4e-807d-338aaef197ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:04.840763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee79eb-e8a2-4d4e-807d-338aaef197ee admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:04.841811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee79eb-e8a2-4d4e-807d-338aaef197ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:04.843930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee79eb-e8a2-4d4e-807d-338aaef197ee admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:04.874816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 982/2019] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:50:04.879480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a06a8-072f-453c-9064-9a77212fad35 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:04.896630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a06a8-072f-453c-9064-9a77212fad35 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:04.924819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc18aa-e548-400a-8a84-65c34c4222f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:04.926044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee79eb-e8a2-4d4e-807d-338aaef197ee admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:04.926660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc18aa-e548-400a-8a84-65c34c4222f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:04.926952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc18aa-e548-400a-8a84-65c34c4222f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:04.927571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc18aa-e548-400a-8a84-65c34c4222f3 admin admin] PATH_INFO: `/v3/projects/192a5c537f2f4724b4849e4a1f163e88/users/c3f3b7740413434ea8dfadd22460ab9b/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:04.928283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2eee79eb-e8a2-4d4e-807d-338aaef197ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:04.951502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee79eb-e8a2-4d4e-807d-338aaef197ee admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:04.961169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:04.962255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:04.962539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:04.962855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] PATH_INFO: `/v3/projects/30e5d0b3f2f4406797a20e38cd04fd95` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:04.965602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30e5d0b3f2f4406797a20e38cd04fd95)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:04.967048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc18aa-e548-400a-8a84-65c34c4222f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192a5c537f2f4724b4849e4a1f163e88, user_id=c3f3b7740413434ea8dfadd22460ab9b, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:04.969518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:04.980668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc18aa-e548-400a-8a84-65c34c4222f3 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:05.012497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1038/2020] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:04 2022] PUT /identity/v3/projects/192a5c537f2f4724b4849e4a1f163e88/users/c3f3b7740413434ea8dfadd22460ab9b/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:05.017262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc18aa-e548-400a-8a84-65c34c4222f3 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:05.038250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30e5d0b3f2f4406797a20e38cd04fd95', 'request_id': 'req-2d46a000-0223-4294-976b-a62fd704c0a0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.045634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30e5d0b3f2f4406797a20e38cd04fd95', 'request_id': 'req-2d46a000-0223-4294-976b-a62fd704c0a0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.052357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30e5d0b3f2f4406797a20e38cd04fd95', 'request_id': 'req-2d46a000-0223-4294-976b-a62fd704c0a0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.055714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:05.072867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] The token cache is being invalidate because project 30e5d0b3f2f4406797a20e38cd04fd95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:05.073176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.074705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.076920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 983/2021] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:04 2022] DELETE /identity/v3/projects/30e5d0b3f2f4406797a20e38cd04fd95 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:05.081056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:05.082483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:05.097799 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc18aa-e548-400a-8a84-65c34c4222f3 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:05.169452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46a000-0223-4294-976b-a62fd704c0a0 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:05.171795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c888fdd-b433-438f-9a75-275536eea7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:05.172442 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c888fdd-b433-438f-9a75-275536eea7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:05.173261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c888fdd-b433-438f-9a75-275536eea7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:05.173621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c888fdd-b433-438f-9a75-275536eea7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:05.174308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c888fdd-b433-438f-9a75-275536eea7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:05.174847 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:05.175722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:05.175897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c888fdd-b433-438f-9a75-275536eea7e4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:05.176000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:05.176259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] PATH_INFO: `/v3/users/a6533a3a2e524b9fa022deb684c864f4` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:05.177518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6533a3a2e524b9fa022deb684c864f4)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:05.179462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:05.187920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1039/2022] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:50:05.192415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c888fdd-b433-438f-9a75-275536eea7e4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:05.194129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0c888fdd-b433-438f-9a75-275536eea7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:05.206478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c888fdd-b433-438f-9a75-275536eea7e4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:05.211947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d9e34-7138-40bd-9e79-24c94994f85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:05.212916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d9e34-7138-40bd-9e79-24c94994f85b admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:05.213274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d9e34-7138-40bd-9e79-24c94994f85b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:05.213589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d9e34-7138-40bd-9e79-24c94994f85b admin admin] PATH_INFO: `/v3/projects/192a5c537f2f4724b4849e4a1f163e88/users/c3f3b7740413434ea8dfadd22460ab9b/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:05.215964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d9e34-7138-40bd-9e79-24c94994f85b admin admin] RBAC: Authorizing `identity:create_grant(project_id=192a5c537f2f4724b4849e4a1f163e88, user_id=c3f3b7740413434ea8dfadd22460ab9b, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:05.219081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d9e34-7138-40bd-9e79-24c94994f85b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:05.221656 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:05.232310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1040/2023] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:05 2022] PUT /identity/v3/projects/192a5c537f2f4724b4849e4a1f163e88/users/c3f3b7740413434ea8dfadd22460ab9b/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:05.233471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6533a3a2e524b9fa022deb684c864f4'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.244068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e671c4-f7b1-4a72-90b9-9e9fd46e011f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:05.244266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6533a3a2e524b9fa022deb684c864f4'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.244361 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e671c4-f7b1-4a72-90b9-9e9fd46e011f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:05.244656 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e671c4-f7b1-4a72-90b9-9e9fd46e011f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:05.252772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6533a3a2e524b9fa022deb684c864f4'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.267390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6533a3a2e524b9fa022deb684c864f4'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.268286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6533a3a2e524b9fa022deb684c864f4'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.274712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-25e671c4-f7b1-4a72-90b9-9e9fd46e011f None None] MFA Rules not processed for user `c3f3b7740413434ea8dfadd22460ab9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:05.275942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-25e671c4-f7b1-4a72-90b9-9e9fd46e011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:05.276250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 984/2024] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:05 2022] DELETE /identity/v3/users/a6533a3a2e524b9fa022deb684c864f4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:05.368762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1041/2025] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:05.377878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:05.380848 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:05.498204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d27abd-9366-4906-aa66-f1577af33891 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:05.508763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f8219-dd52-41be-a14f-8a7a0a52c092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:05.509483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f8219-dd52-41be-a14f-8a7a0a52c092 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:05.509764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f8219-dd52-41be-a14f-8a7a0a52c092 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:05.512776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f8219-dd52-41be-a14f-8a7a0a52c092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:05.544112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f8219-dd52-41be-a14f-8a7a0a52c092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:05.547740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f8219-dd52-41be-a14f-8a7a0a52c092 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:05.581036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 985/2026] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:05.623269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e671c4-f7b1-4a72-90b9-9e9fd46e011f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:05.626214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-25e671c4-f7b1-4a72-90b9-9e9fd46e011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:05.735986 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e671c4-f7b1-4a72-90b9-9e9fd46e011f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:05.745152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94b6ce-0cfc-4ae3-a2a8-107e0e7b9907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:05.746264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94b6ce-0cfc-4ae3-a2a8-107e0e7b9907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:05.746551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94b6ce-0cfc-4ae3-a2a8-107e0e7b9907 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:05.746982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94b6ce-0cfc-4ae3-a2a8-107e0e7b9907 admin admin] PATH_INFO: `/v3/users/8927179bd32842fb90fe6d9c7b903aca` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:05.748225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94b6ce-0cfc-4ae3-a2a8-107e0e7b9907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8927179bd32842fb90fe6d9c7b903aca)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:05.750277 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94b6ce-0cfc-4ae3-a2a8-107e0e7b9907 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:05.805843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1d94b6ce-0cfc-4ae3-a2a8-107e0e7b9907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8927179bd32842fb90fe6d9c7b903aca'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.834983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1d94b6ce-0cfc-4ae3-a2a8-107e0e7b9907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8927179bd32842fb90fe6d9c7b903aca'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.845688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1d94b6ce-0cfc-4ae3-a2a8-107e0e7b9907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8927179bd32842fb90fe6d9c7b903aca'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.846758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1d94b6ce-0cfc-4ae3-a2a8-107e0e7b9907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8927179bd32842fb90fe6d9c7b903aca'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.851929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f8219-dd52-41be-a14f-8a7a0a52c092 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:05.854991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-542f8219-dd52-41be-a14f-8a7a0a52c092 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:05.857986 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1d94b6ce-0cfc-4ae3-a2a8-107e0e7b9907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8927179bd32842fb90fe6d9c7b903aca'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:05.860023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:05.872152 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1042/2027] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:05 2022] DELETE /identity/v3/users/8927179bd32842fb90fe6d9c7b903aca => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:05.874584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772cdb2-483c-4b84-8092-cd0bc2d49181 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:05.875655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f8219-dd52-41be-a14f-8a7a0a52c092 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:05.875976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772cdb2-483c-4b84-8092-cd0bc2d49181 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:05.876279 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772cdb2-483c-4b84-8092-cd0bc2d49181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:05.893493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:05.896858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-2772cdb2-483c-4b84-8092-cd0bc2d49181 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:05.898208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2772cdb2-483c-4b84-8092-cd0bc2d49181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:05.975208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:05.976157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:05.976530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:05.976820 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] PATH_INFO: `/v3/projects/df5282f89e4f4ce08f9b7f6e15ca08cd` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:05.978400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5282f89e4f4ce08f9b7f6e15ca08cd)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:05.980560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:05.989512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1043/2028] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:06.002765 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772cdb2-483c-4b84-8092-cd0bc2d49181 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:06.004195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2772cdb2-483c-4b84-8092-cd0bc2d49181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:06.053693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5282f89e4f4ce08f9b7f6e15ca08cd', 'request_id': 'req-b2377ec9-76e7-481d-8780-755e41fa61fd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.060003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5282f89e4f4ce08f9b7f6e15ca08cd', 'request_id': 'req-b2377ec9-76e7-481d-8780-755e41fa61fd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.067866 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5282f89e4f4ce08f9b7f6e15ca08cd', 'request_id': 'req-b2377ec9-76e7-481d-8780-755e41fa61fd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.070578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772cdb2-483c-4b84-8092-cd0bc2d49181 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:06.081719 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] The token cache is being invalidate because project df5282f89e4f4ce08f9b7f6e15ca08cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:06.082116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.082809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b2377ec9-76e7-481d-8780-755e41fa61fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.085318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 986/2029] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:05 2022] DELETE /identity/v3/projects/df5282f89e4f4ce08f9b7f6e15ca08cd => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:06.091927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbf0d6-c583-42c2-a41c-cf585395bb17 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.092223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbf0d6-c583-42c2-a41c-cf585395bb17 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.092515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbf0d6-c583-42c2-a41c-cf585395bb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.112296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-3fbbf0d6-c583-42c2-a41c-cf585395bb17 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:06.113571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3fbbf0d6-c583-42c2-a41c-cf585395bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:06.134419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22828d-bf6b-49f6-ab0d-1582fa712bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:06.134996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22828d-bf6b-49f6-ab0d-1582fa712bac service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.135285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22828d-bf6b-49f6-ab0d-1582fa712bac service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.135568 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22828d-bf6b-49f6-ab0d-1582fa712bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.137397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-cb22828d-bf6b-49f6-ab0d-1582fa712bac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:06.183365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 987/2030] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:06.192196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c241195-d43e-4297-b032-e5dda359a708 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.192493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c241195-d43e-4297-b032-e5dda359a708 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.192777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c241195-d43e-4297-b032-e5dda359a708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.213241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22828d-bf6b-49f6-ab0d-1582fa712bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:06.214953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22828d-bf6b-49f6-ab0d-1582fa712bac service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:06.215623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-3c241195-d43e-4297-b032-e5dda359a708 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:06.216876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3c241195-d43e-4297-b032-e5dda359a708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:06.225412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 988/2031] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:06.225702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1044/2032] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:50:06.229910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c241195-d43e-4297-b032-e5dda359a708 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:06.239172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c241195-d43e-4297-b032-e5dda359a708 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:06.244164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cf011-1eb1-4c88-885a-0749e240cd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:06.244934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cf011-1eb1-4c88-885a-0749e240cd87 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.245201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cf011-1eb1-4c88-885a-0749e240cd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.245597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cf011-1eb1-4c88-885a-0749e240cd87 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.246395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cf011-1eb1-4c88-885a-0749e240cd87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:06.246709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cf011-1eb1-4c88-885a-0749e240cd87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:06.248350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cf011-1eb1-4c88-885a-0749e240cd87 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:06.261512 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 989/2033] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:50:06.265673 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22828d-bf6b-49f6-ab0d-1582fa712bac service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:06.272901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:06.277777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22828d-bf6b-49f6-ab0d-1582fa712bac service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:06.283353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46dea7-d8a4-4fc3-a418-dacaf2b533c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:06.284224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46dea7-d8a4-4fc3-a418-dacaf2b533c8 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.284881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46dea7-d8a4-4fc3-a418-dacaf2b533c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.284881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46dea7-d8a4-4fc3-a418-dacaf2b533c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.285616 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46dea7-d8a4-4fc3-a418-dacaf2b533c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:06.288021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46dea7-d8a4-4fc3-a418-dacaf2b533c8 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:06.318496 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1045/2034] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:06 2022] POST /identity/v3/projects => generated 401 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:50:06.322548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cf011-1eb1-4c88-885a-0749e240cd87 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:06.333816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cf011-1eb1-4c88-885a-0749e240cd87 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:06.359762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:06.406244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075f1a-73e5-424f-be53-0af7799924b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:06.406961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075f1a-73e5-424f-be53-0af7799924b8 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.407257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075f1a-73e5-424f-be53-0af7799924b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.407534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075f1a-73e5-424f-be53-0af7799924b8 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.408249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b075f1a-73e5-424f-be53-0af7799924b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:06.410277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b075f1a-73e5-424f-be53-0af7799924b8 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:06.436040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 990/2035] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:06 2022] POST /identity/v3/users => generated 301 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:06.437515 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46dea7-d8a4-4fc3-a418-dacaf2b533c8 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:06.452104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46dea7-d8a4-4fc3-a418-dacaf2b533c8 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:06.457688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf3b1f-4f72-42e3-b19f-0bc83a1366c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:06.458475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf3b1f-4f72-42e3-b19f-0bc83a1366c2 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.458804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf3b1f-4f72-42e3-b19f-0bc83a1366c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.459153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf3b1f-4f72-42e3-b19f-0bc83a1366c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.459872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf3b1f-4f72-42e3-b19f-0bc83a1366c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:06.461723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf3b1f-4f72-42e3-b19f-0bc83a1366c2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:06.473350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1046/2036] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:06.477456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075f1a-73e5-424f-be53-0af7799924b8 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:06.491940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075f1a-73e5-424f-be53-0af7799924b8 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:06.498881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bbe0d-f085-42ee-91ad-98ede66b0949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:06.499808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bbe0d-f085-42ee-91ad-98ede66b0949 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.500242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bbe0d-f085-42ee-91ad-98ede66b0949 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.500534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bbe0d-f085-42ee-91ad-98ede66b0949 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.501889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6bbe0d-f085-42ee-91ad-98ede66b0949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:06.503755 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6bbe0d-f085-42ee-91ad-98ede66b0949 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:06.521077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 991/2037] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:06.522976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf3b1f-4f72-42e3-b19f-0bc83a1366c2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:06.523993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bbe0d-f085-42ee-91ad-98ede66b0949 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:06.526786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8b6bbe0d-f085-42ee-91ad-98ede66b0949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:06.536166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf3b1f-4f72-42e3-b19f-0bc83a1366c2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:06.541032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bbe0d-f085-42ee-91ad-98ede66b0949 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:06.545699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:06.546585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.546855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.547132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] PATH_INFO: `/v3/projects/32b877071c79438292e23af5f0401e6c` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.548934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=32b877071c79438292e23af5f0401e6c)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:06.550489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839011b-1e34-466e-a8fa-43c7357c63d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:06.551562 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839011b-1e34-466e-a8fa-43c7357c63d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.551852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:06.551968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839011b-1e34-466e-a8fa-43c7357c63d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.552265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839011b-1e34-466e-a8fa-43c7357c63d2 admin admin] PATH_INFO: `/v3/projects/4091c0c016c8466cadf3bcf92cb21030/users/3dec8074e5f04e5caa18901d1ce0bc66/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.577543 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839011b-1e34-466e-a8fa-43c7357c63d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4091c0c016c8466cadf3bcf92cb21030, user_id=3dec8074e5f04e5caa18901d1ce0bc66, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:06.580731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839011b-1e34-466e-a8fa-43c7357c63d2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:06.609965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1047/2038] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:06 2022] PUT /identity/v3/projects/4091c0c016c8466cadf3bcf92cb21030/users/3dec8074e5f04e5caa18901d1ce0bc66/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:06.612294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8916dc-6579-43b5-aecb-51400c7deb54 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.612597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8916dc-6579-43b5-aecb-51400c7deb54 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.613319 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8916dc-6579-43b5-aecb-51400c7deb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.624543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32b877071c79438292e23af5f0401e6c', 'request_id': 'req-119cbeb2-d0a0-4188-8fff-ffe841564b6c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.635331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32b877071c79438292e23af5f0401e6c', 'request_id': 'req-119cbeb2-d0a0-4188-8fff-ffe841564b6c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.643161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32b877071c79438292e23af5f0401e6c', 'request_id': 'req-119cbeb2-d0a0-4188-8fff-ffe841564b6c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.655131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-4a8916dc-6579-43b5-aecb-51400c7deb54 None None] MFA Rules not processed for user `3dec8074e5f04e5caa18901d1ce0bc66`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:06.656364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4a8916dc-6579-43b5-aecb-51400c7deb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:06.659278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] The token cache is being invalidate because project 32b877071c79438292e23af5f0401e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:06.659753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.660427 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.662433 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 992/2039] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:06 2022] DELETE /identity/v3/projects/32b877071c79438292e23af5f0401e6c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:06.666655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:06.671509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:06.729829 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1048/2040] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:06.734351 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8916dc-6579-43b5-aecb-51400c7deb54 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:06.736755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4a8916dc-6579-43b5-aecb-51400c7deb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:06.760183 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cbeb2-d0a0-4188-8fff-ffe841564b6c admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:06.766462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:06.770083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.770083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.770083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] PATH_INFO: `/v3/users/c3f3b7740413434ea8dfadd22460ab9b` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.770083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f3b7740413434ea8dfadd22460ab9b)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:06.771930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:06.818360 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8916dc-6579-43b5-aecb-51400c7deb54 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:06.820531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:06.824229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411003ef-55f5-43a1-b925-0e7ab61c6cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:06.824980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411003ef-55f5-43a1-b925-0e7ab61c6cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:06.825347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411003ef-55f5-43a1-b925-0e7ab61c6cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:06.825729 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411003ef-55f5-43a1-b925-0e7ab61c6cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:06.826493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411003ef-55f5-43a1-b925-0e7ab61c6cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:06.828472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411003ef-55f5-43a1-b925-0e7ab61c6cc6 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:06.838692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f3b7740413434ea8dfadd22460ab9b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.848755 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f3b7740413434ea8dfadd22460ab9b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.858323 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f3b7740413434ea8dfadd22460ab9b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.869589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1049/2041] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:06 2022] POST /identity/v3/projects => generated 401 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:06.872642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411003ef-55f5-43a1-b925-0e7ab61c6cc6 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:06.876411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f3b7740413434ea8dfadd22460ab9b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.877407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f3b7740413434ea8dfadd22460ab9b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:06.885573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 993/2042] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:06 2022] DELETE /identity/v3/users/c3f3b7740413434ea8dfadd22460ab9b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:06.909492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:06.949701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411003ef-55f5-43a1-b925-0e7ab61c6cc6 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.040433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687b88d-69d4-4af3-a4be-3a8069ee73cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.041425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687b88d-69d4-4af3-a4be-3a8069ee73cf admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.041710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687b88d-69d4-4af3-a4be-3a8069ee73cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.042013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687b88d-69d4-4af3-a4be-3a8069ee73cf admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.042702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687b88d-69d4-4af3-a4be-3a8069ee73cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.044841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687b88d-69d4-4af3-a4be-3a8069ee73cf admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.071115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1050/2043] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:06 2022] POST /identity/v3/users => generated 307 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:07.075066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.076574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:07.091331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cd815-50b8-4401-93ed-987cb3d027fa admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.098388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800232c-6fbf-4ac8-a294-ecbe92bf72fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.099291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800232c-6fbf-4ac8-a294-ecbe92bf72fc admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.099586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800232c-6fbf-4ac8-a294-ecbe92bf72fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.099865 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800232c-6fbf-4ac8-a294-ecbe92bf72fc admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.100601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800232c-6fbf-4ac8-a294-ecbe92bf72fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.102273 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800232c-6fbf-4ac8-a294-ecbe92bf72fc admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.114045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 994/2044] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:50:07.118533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687b88d-69d4-4af3-a4be-3a8069ee73cf admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.136032 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687b88d-69d4-4af3-a4be-3a8069ee73cf admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.142286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9891e-dca1-4e21-a40b-73c3e57153e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.143693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9891e-dca1-4e21-a40b-73c3e57153e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.144079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9891e-dca1-4e21-a40b-73c3e57153e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.144533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9891e-dca1-4e21-a40b-73c3e57153e4 admin admin] PATH_INFO: `/v3/projects/8507215fa9564a1c8365c782a146527a/users/fd8d23c5a1ff406eaa56444832e29c2b/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.162728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9891e-dca1-4e21-a40b-73c3e57153e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8507215fa9564a1c8365c782a146527a, user_id=fd8d23c5a1ff406eaa56444832e29c2b, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.167125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9891e-dca1-4e21-a40b-73c3e57153e4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.194700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1051/2045] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:07 2022] PUT /identity/v3/projects/8507215fa9564a1c8365c782a146527a/users/fd8d23c5a1ff406eaa56444832e29c2b/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:07.201832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800232c-6fbf-4ac8-a294-ecbe92bf72fc admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.217303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800232c-6fbf-4ac8-a294-ecbe92bf72fc admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.279525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9891e-dca1-4e21-a40b-73c3e57153e4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.281776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-92d9891e-dca1-4e21-a40b-73c3e57153e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:07.290373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9462112-1b00-4756-a3f5-a12190bd2a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.291070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9462112-1b00-4756-a3f5-a12190bd2a00 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.291362 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9462112-1b00-4756-a3f5-a12190bd2a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.291645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9462112-1b00-4756-a3f5-a12190bd2a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.292398 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9462112-1b00-4756-a3f5-a12190bd2a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.294070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9462112-1b00-4756-a3f5-a12190bd2a00 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.306321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 995/2046] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:07.310417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9462112-1b00-4756-a3f5-a12190bd2a00 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.321516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9462112-1b00-4756-a3f5-a12190bd2a00 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.327000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50511fc2-e38c-4fe8-ae91-c5b675f692e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.327900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50511fc2-e38c-4fe8-ae91-c5b675f692e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.328191 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50511fc2-e38c-4fe8-ae91-c5b675f692e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.328476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50511fc2-e38c-4fe8-ae91-c5b675f692e2 admin admin] PATH_INFO: `/v3/projects/8507215fa9564a1c8365c782a146527a/users/fd8d23c5a1ff406eaa56444832e29c2b/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.330496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50511fc2-e38c-4fe8-ae91-c5b675f692e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8507215fa9564a1c8365c782a146527a, user_id=fd8d23c5a1ff406eaa56444832e29c2b, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.333163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50511fc2-e38c-4fe8-ae91-c5b675f692e2 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.344702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 996/2047] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:07 2022] PUT /identity/v3/projects/8507215fa9564a1c8365c782a146527a/users/fd8d23c5a1ff406eaa56444832e29c2b/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:07.353537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9891e-dca1-4e21-a40b-73c3e57153e4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.356044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9789a6-3ccb-4a16-9832-b1f8b4ea1f49 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.356334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9789a6-3ccb-4a16-9832-b1f8b4ea1f49 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.356619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9789a6-3ccb-4a16-9832-b1f8b4ea1f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.386193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0a9789a6-3ccb-4a16-9832-b1f8b4ea1f49 None None] MFA Rules not processed for user `fd8d23c5a1ff406eaa56444832e29c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:07.387359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0a9789a6-3ccb-4a16-9832-b1f8b4ea1f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:07.402773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:07.424177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.425080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.425383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.425678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] PATH_INFO: `/v3/projects/192a5c537f2f4724b4849e4a1f163e88` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.426904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] RBAC: Authorizing `identity:delete_project(project_id=192a5c537f2f4724b4849e4a1f163e88)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.428816 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.461078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 997/2048] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:07 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:07.472983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9789a6-3ccb-4a16-9832-b1f8b4ea1f49 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.474465 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0a9789a6-3ccb-4a16-9832-b1f8b4ea1f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:07.479487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192a5c537f2f4724b4849e4a1f163e88', 'request_id': 'req-288798df-62b1-4165-bcf9-805788600ede'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:07.480740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:07.487336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192a5c537f2f4724b4849e4a1f163e88', 'request_id': 'req-288798df-62b1-4165-bcf9-805788600ede'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:07.493875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192a5c537f2f4724b4849e4a1f163e88', 'request_id': 'req-288798df-62b1-4165-bcf9-805788600ede'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:07.506115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] The token cache is being invalidate because project 192a5c537f2f4724b4849e4a1f163e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:07.506412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:07.507758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-288798df-62b1-4165-bcf9-805788600ede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:07.509750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1052/2049] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:07 2022] DELETE /identity/v3/projects/192a5c537f2f4724b4849e4a1f163e88 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:07.515766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b317d-80c0-456a-975e-48192d28977c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.516061 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b317d-80c0-456a-975e-48192d28977c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.516379 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b317d-80c0-456a-975e-48192d28977c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.534880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-ad3b317d-80c0-456a-975e-48192d28977c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:07.536206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ad3b317d-80c0-456a-975e-48192d28977c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:07.539022 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9789a6-3ccb-4a16-9832-b1f8b4ea1f49 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.602873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd562b1-7bb9-47d8-8c2c-91bba64aabc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.603538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd562b1-7bb9-47d8-8c2c-91bba64aabc5 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.603830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd562b1-7bb9-47d8-8c2c-91bba64aabc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.604114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd562b1-7bb9-47d8-8c2c-91bba64aabc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.604251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1053/2050] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:07.606099 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8dd562b1-7bb9-47d8-8c2c-91bba64aabc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:07.613459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642113aa-fe98-4e4d-b52e-b6785b0ba2e7 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.613754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642113aa-fe98-4e4d-b52e-b6785b0ba2e7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.614197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642113aa-fe98-4e4d-b52e-b6785b0ba2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.638402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-642113aa-fe98-4e4d-b52e-b6785b0ba2e7 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:07.638402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-642113aa-fe98-4e4d-b52e-b6785b0ba2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:07.643964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1054/2051] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:07.648171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642113aa-fe98-4e4d-b52e-b6785b0ba2e7 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.658166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642113aa-fe98-4e4d-b52e-b6785b0ba2e7 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.663153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fca047-6bf4-4d55-bb35-7cefe19e2272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.663879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fca047-6bf4-4d55-bb35-7cefe19e2272 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.664236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fca047-6bf4-4d55-bb35-7cefe19e2272 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.664759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fca047-6bf4-4d55-bb35-7cefe19e2272 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.665601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fca047-6bf4-4d55-bb35-7cefe19e2272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:07.667299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fca047-6bf4-4d55-bb35-7cefe19e2272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.668918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fca047-6bf4-4d55-bb35-7cefe19e2272 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.678219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd562b1-7bb9-47d8-8c2c-91bba64aabc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.679687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd562b1-7bb9-47d8-8c2c-91bba64aabc5 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.684009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1055/2052] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:50:07.688989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fca047-6bf4-4d55-bb35-7cefe19e2272 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.691447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 998/2053] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:07 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:07.700719 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fca047-6bf4-4d55-bb35-7cefe19e2272 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.705532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03834473-e561-429b-91c5-b079cb10b152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.706202 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03834473-e561-429b-91c5-b079cb10b152 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.706496 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03834473-e561-429b-91c5-b079cb10b152 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.706775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03834473-e561-429b-91c5-b079cb10b152 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.707471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03834473-e561-429b-91c5-b079cb10b152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.709455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03834473-e561-429b-91c5-b079cb10b152 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.745101 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1056/2054] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:07 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:07.746293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd562b1-7bb9-47d8-8c2c-91bba64aabc5 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.759656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd562b1-7bb9-47d8-8c2c-91bba64aabc5 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.774471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03834473-e561-429b-91c5-b079cb10b152 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.776286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-03834473-e561-429b-91c5-b079cb10b152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:07.826618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:07.832519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a7015-7057-4a71-ae56-c49e13ec7383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.833201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a7015-7057-4a71-ae56-c49e13ec7383 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.833670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a7015-7057-4a71-ae56-c49e13ec7383 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.833949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a7015-7057-4a71-ae56-c49e13ec7383 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.834658 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a7015-7057-4a71-ae56-c49e13ec7383 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.837021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a7015-7057-4a71-ae56-c49e13ec7383 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.859642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:07.869747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 999/2055] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:07 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:07.870812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03834473-e561-429b-91c5-b079cb10b152 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.873507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a7015-7057-4a71-ae56-c49e13ec7383 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.877572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bba6d-fbf7-4d58-b546-0dd2759ef846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.878595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bba6d-fbf7-4d58-b546-0dd2759ef846 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.879053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bba6d-fbf7-4d58-b546-0dd2759ef846 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.879489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bba6d-fbf7-4d58-b546-0dd2759ef846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.885731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9b5bba6d-fbf7-4d58-b546-0dd2759ef846 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:07.889820 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a7015-7057-4a71-ae56-c49e13ec7383 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.895746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661141c-e3d2-4d37-a165-3f17ae0349ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.896506 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661141c-e3d2-4d37-a165-3f17ae0349ba admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.896796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661141c-e3d2-4d37-a165-3f17ae0349ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.897072 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661141c-e3d2-4d37-a165-3f17ae0349ba admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.897873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661141c-e3d2-4d37-a165-3f17ae0349ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.899492 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661141c-e3d2-4d37-a165-3f17ae0349ba admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.911476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1000/2056] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:50:07.915650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661141c-e3d2-4d37-a165-3f17ae0349ba admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.928237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661141c-e3d2-4d37-a165-3f17ae0349ba admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.934489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57c1e3-9c0b-4113-bddd-1fbb5b1e5c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.935275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57c1e3-9c0b-4113-bddd-1fbb5b1e5c60 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.935569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57c1e3-9c0b-4113-bddd-1fbb5b1e5c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.935908 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57c1e3-9c0b-4113-bddd-1fbb5b1e5c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.936577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57c1e3-9c0b-4113-bddd-1fbb5b1e5c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.938275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57c1e3-9c0b-4113-bddd-1fbb5b1e5c60 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.951388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1001/2057] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:50:07.956049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57c1e3-9c0b-4113-bddd-1fbb5b1e5c60 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:07.966389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57c1e3-9c0b-4113-bddd-1fbb5b1e5c60 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:07.972093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d52f67-dd5a-4fac-a733-594647255fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:07.974240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d52f67-dd5a-4fac-a733-594647255fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:07.974240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d52f67-dd5a-4fac-a733-594647255fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:07.974836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d52f67-dd5a-4fac-a733-594647255fc1 admin admin] PATH_INFO: `/v3/projects/1d4af75d3f844b1ab829af204f864ce3/users/38a2ba51d651497a92bdf1ca5782605a/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:07.983838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bba6d-fbf7-4d58-b546-0dd2759ef846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:07.986484 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bba6d-fbf7-4d58-b546-0dd2759ef846 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:07.996846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1057/2058] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:07.998813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d52f67-dd5a-4fac-a733-594647255fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4af75d3f844b1ab829af204f864ce3, user_id=38a2ba51d651497a92bdf1ca5782605a, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:08.001960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d52f67-dd5a-4fac-a733-594647255fc1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:08.019378 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1002/2059] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:07 2022] PUT /identity/v3/projects/1d4af75d3f844b1ab829af204f864ce3/users/38a2ba51d651497a92bdf1ca5782605a/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:08.031601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ae9fd-65b2-485c-8b7e-76887ccc1456 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:08.032797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ae9fd-65b2-485c-8b7e-76887ccc1456 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:08.033094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ae9fd-65b2-485c-8b7e-76887ccc1456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:08.077276 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-d51ae9fd-65b2-485c-8b7e-76887ccc1456 None None] MFA Rules not processed for user `38a2ba51d651497a92bdf1ca5782605a`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:08.079380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d51ae9fd-65b2-485c-8b7e-76887ccc1456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:08.165631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1058/2060] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:08.170792 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d52f67-dd5a-4fac-a733-594647255fc1 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:08.250765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d52f67-dd5a-4fac-a733-594647255fc1 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:08.257405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860be0e2-578e-4692-a26f-ef1691e6fe81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:08.258732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860be0e2-578e-4692-a26f-ef1691e6fe81 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:08.258883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860be0e2-578e-4692-a26f-ef1691e6fe81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:08.259304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860be0e2-578e-4692-a26f-ef1691e6fe81 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:08.260159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860be0e2-578e-4692-a26f-ef1691e6fe81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:08.262672 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860be0e2-578e-4692-a26f-ef1691e6fe81 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:08.294554 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1003/2061] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:08 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:08.298381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ae9fd-65b2-485c-8b7e-76887ccc1456 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:08.328004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ae9fd-65b2-485c-8b7e-76887ccc1456 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:08.425790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c5670-7c47-49d8-9403-fda0f3f71e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:08.426508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c5670-7c47-49d8-9403-fda0f3f71e2f admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:08.426796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c5670-7c47-49d8-9403-fda0f3f71e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:08.427077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c5670-7c47-49d8-9403-fda0f3f71e2f admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:08.427795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c5670-7c47-49d8-9403-fda0f3f71e2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:08.430280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c5670-7c47-49d8-9403-fda0f3f71e2f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:08.456010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1059/2062] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:08 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:08.460120 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860be0e2-578e-4692-a26f-ef1691e6fe81 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:08.471043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860be0e2-578e-4692-a26f-ef1691e6fe81 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:08.476244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa2576-331d-444e-96f1-beab90bae5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:08.476975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa2576-331d-444e-96f1-beab90bae5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:08.477438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa2576-331d-444e-96f1-beab90bae5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:08.477753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa2576-331d-444e-96f1-beab90bae5c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:08.478744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa2576-331d-444e-96f1-beab90bae5c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:08.480577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa2576-331d-444e-96f1-beab90bae5c7 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:08.491854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1004/2063] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:08.496023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c5670-7c47-49d8-9403-fda0f3f71e2f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:08.502993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:08.506852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c5670-7c47-49d8-9403-fda0f3f71e2f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:08.512560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb03d5-6db1-444c-9b18-446814627334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:08.513491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb03d5-6db1-444c-9b18-446814627334 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:08.513773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb03d5-6db1-444c-9b18-446814627334 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:08.514049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb03d5-6db1-444c-9b18-446814627334 admin admin] PATH_INFO: `/v3/projects/5491cb781f944faa861e99c880429338/users/c98fa83b32134c6b805d1b9dd83c309a/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:08.529510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb03d5-6db1-444c-9b18-446814627334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5491cb781f944faa861e99c880429338, user_id=c98fa83b32134c6b805d1b9dd83c309a, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:08.532175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb03d5-6db1-444c-9b18-446814627334 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:08.545210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1060/2064] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:08 2022] PUT /identity/v3/projects/5491cb781f944faa861e99c880429338/users/c98fa83b32134c6b805d1b9dd83c309a/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:08.549175 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa2576-331d-444e-96f1-beab90bae5c7 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:08.559226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa2576-331d-444e-96f1-beab90bae5c7 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:08.619619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f90e2-5c22-4ba5-bf51-dcb30ee087fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:08.620294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f90e2-5c22-4ba5-bf51-dcb30ee087fc admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:08.620577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f90e2-5c22-4ba5-bf51-dcb30ee087fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:08.620852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f90e2-5c22-4ba5-bf51-dcb30ee087fc admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:08.621735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f90e2-5c22-4ba5-bf51-dcb30ee087fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:08.623333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f90e2-5c22-4ba5-bf51-dcb30ee087fc admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:08.636864 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1005/2065] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:08.642013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb03d5-6db1-444c-9b18-446814627334 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:08.652548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb03d5-6db1-444c-9b18-446814627334 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:08.658106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a0bfb-bf57-42b2-ade0-697be7ceb54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:08.659003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a0bfb-bf57-42b2-ade0-697be7ceb54a admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:08.659294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a0bfb-bf57-42b2-ade0-697be7ceb54a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:08.659573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a0bfb-bf57-42b2-ade0-697be7ceb54a admin admin] PATH_INFO: `/v3/projects/5491cb781f944faa861e99c880429338/users/c98fa83b32134c6b805d1b9dd83c309a/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:08.661656 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a0bfb-bf57-42b2-ade0-697be7ceb54a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5491cb781f944faa861e99c880429338, user_id=c98fa83b32134c6b805d1b9dd83c309a, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:08.664315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a0bfb-bf57-42b2-ade0-697be7ceb54a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:08.676295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1061/2066] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:08 2022] PUT /identity/v3/projects/5491cb781f944faa861e99c880429338/users/c98fa83b32134c6b805d1b9dd83c309a/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:08.687318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa561643-086c-4858-a6ed-ff1dcf512f45 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:08.687609 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa561643-086c-4858-a6ed-ff1dcf512f45 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:08.687892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa561643-086c-4858-a6ed-ff1dcf512f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:08.715596 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-aa561643-086c-4858-a6ed-ff1dcf512f45 None None] MFA Rules not processed for user `c98fa83b32134c6b805d1b9dd83c309a`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:08.717131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-aa561643-086c-4858-a6ed-ff1dcf512f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:08.731801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:08.795930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1006/2067] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:08.809311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a0bfb-bf57-42b2-ade0-697be7ceb54a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:08.811357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b98a0bfb-bf57-42b2-ade0-697be7ceb54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:08.874528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a0bfb-bf57-42b2-ade0-697be7ceb54a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:08.878551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c942a-c9f1-40fb-a4ed-e47e16b26070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:08.879207 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c942a-c9f1-40fb-a4ed-e47e16b26070 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:08.879488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c942a-c9f1-40fb-a4ed-e47e16b26070 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:08.879765 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c942a-c9f1-40fb-a4ed-e47e16b26070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:08.890419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c942a-c9f1-40fb-a4ed-e47e16b26070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:08.892498 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c942a-c9f1-40fb-a4ed-e47e16b26070 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:08.902846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1062/2068] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:50:08.997426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa561643-086c-4858-a6ed-ff1dcf512f45 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:09.008294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa561643-086c-4858-a6ed-ff1dcf512f45 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:09.013585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500bef5-7e20-4030-9653-99ec0f329068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:09.014722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500bef5-7e20-4030-9653-99ec0f329068 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:09.015227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500bef5-7e20-4030-9653-99ec0f329068 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:09.015737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500bef5-7e20-4030-9653-99ec0f329068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:09.091649 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500bef5-7e20-4030-9653-99ec0f329068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:09.093400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500bef5-7e20-4030-9653-99ec0f329068 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:09.104788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1007/2069] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:09.402813 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c942a-c9f1-40fb-a4ed-e47e16b26070 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:09.467623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c942a-c9f1-40fb-a4ed-e47e16b26070 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:09.472910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:09.473880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:09.474160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:09.474558 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] PATH_INFO: `/v3/users/38a2ba51d651497a92bdf1ca5782605a` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:09.475759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] RBAC: Authorizing `identity:delete_user(user_id=38a2ba51d651497a92bdf1ca5782605a)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:09.477698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:09.527400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38a2ba51d651497a92bdf1ca5782605a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:09.540338 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38a2ba51d651497a92bdf1ca5782605a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:09.547456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38a2ba51d651497a92bdf1ca5782605a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:09.548743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38a2ba51d651497a92bdf1ca5782605a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:09.550042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:09.557530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38a2ba51d651497a92bdf1ca5782605a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:09.564504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1063/2070] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:09 2022] DELETE /identity/v3/users/38a2ba51d651497a92bdf1ca5782605a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:09.569043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bee23-be72-4420-94f9-d5fa4169279e None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:09.569435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bee23-be72-4420-94f9-d5fa4169279e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:09.570205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bee23-be72-4420-94f9-d5fa4169279e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:09.588586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-1d2bee23-be72-4420-94f9-d5fa4169279e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:09.589862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1d2bee23-be72-4420-94f9-d5fa4169279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:09.650859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1008/2071] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:09.660758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:09.662188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:09.718958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b13fa-39e7-4fe6-bbf1-5897de65d91c admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:09.722847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23383437-2f96-4058-a4d0-e2b68d703ac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:09.723485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23383437-2f96-4058-a4d0-e2b68d703ac9 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:09.723765 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23383437-2f96-4058-a4d0-e2b68d703ac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:09.724038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23383437-2f96-4058-a4d0-e2b68d703ac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:09.733538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23383437-2f96-4058-a4d0-e2b68d703ac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:09.735633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23383437-2f96-4058-a4d0-e2b68d703ac9 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:09.745173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1064/2072] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:10.039407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bee23-be72-4420-94f9-d5fa4169279e None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:10.040919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1d2bee23-be72-4420-94f9-d5fa4169279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:10.053685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bee23-be72-4420-94f9-d5fa4169279e None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:10.058531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:10.059425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:10.059709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:10.060070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] PATH_INFO: `/v3/projects/1d4af75d3f844b1ab829af204f864ce3` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:10.061259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4af75d3f844b1ab829af204f864ce3)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:10.063682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:10.108041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:10.112558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4af75d3f844b1ab829af204f864ce3', 'request_id': 'req-40f9d59c-7be5-40cd-9fe0-11375c4baee4'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.118790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4af75d3f844b1ab829af204f864ce3', 'request_id': 'req-40f9d59c-7be5-40cd-9fe0-11375c4baee4'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.124338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4af75d3f844b1ab829af204f864ce3', 'request_id': 'req-40f9d59c-7be5-40cd-9fe0-11375c4baee4'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.136517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] The token cache is being invalidate because project 1d4af75d3f844b1ab829af204f864ce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:10.136799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.137442 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.138979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1009/2073] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:10 2022] DELETE /identity/v3/projects/1d4af75d3f844b1ab829af204f864ce3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:10.143080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23383437-2f96-4058-a4d0-e2b68d703ac9 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:10.152889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23383437-2f96-4058-a4d0-e2b68d703ac9 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:10.216131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d82915-a958-4b49-84b7-3dd55a3a8b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:10.217021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d82915-a958-4b49-84b7-3dd55a3a8b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:10.217309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d82915-a958-4b49-84b7-3dd55a3a8b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:10.217791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d82915-a958-4b49-84b7-3dd55a3a8b2d admin admin] PATH_INFO: `/v3/users/c98fa83b32134c6b805d1b9dd83c309a` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:10.219272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d82915-a958-4b49-84b7-3dd55a3a8b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c98fa83b32134c6b805d1b9dd83c309a)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:10.221216 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d82915-a958-4b49-84b7-3dd55a3a8b2d admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:10.272337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-51d82915-a958-4b49-84b7-3dd55a3a8b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c98fa83b32134c6b805d1b9dd83c309a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.293822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-51d82915-a958-4b49-84b7-3dd55a3a8b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c98fa83b32134c6b805d1b9dd83c309a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.302404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-51d82915-a958-4b49-84b7-3dd55a3a8b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c98fa83b32134c6b805d1b9dd83c309a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.303050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-51d82915-a958-4b49-84b7-3dd55a3a8b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c98fa83b32134c6b805d1b9dd83c309a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.310949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-51d82915-a958-4b49-84b7-3dd55a3a8b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c98fa83b32134c6b805d1b9dd83c309a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.318480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1065/2074] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:10 2022] DELETE /identity/v3/users/c98fa83b32134c6b805d1b9dd83c309a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:10.703485 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:10.704956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:10.718400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9d59c-7be5-40cd-9fe0-11375c4baee4 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:10.787963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:10.788606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:10.788904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:10.789180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] PATH_INFO: `/v3/projects/5491cb781f944faa861e99c880429338` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:10.790855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=5491cb781f944faa861e99c880429338)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:10.792845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:10.850442 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5491cb781f944faa861e99c880429338', 'request_id': 'req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.856593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5491cb781f944faa861e99c880429338', 'request_id': 'req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.865844 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5491cb781f944faa861e99c880429338', 'request_id': 'req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.879526 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] The token cache is being invalidate because project 5491cb781f944faa861e99c880429338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:10.879858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.880498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-f1d5aca4-8095-4e96-9b6e-e1ff282a0dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:10.882325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1010/2075] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:10 2022] DELETE /identity/v3/projects/5491cb781f944faa861e99c880429338 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:10.894333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6eebd-75b5-4570-af54-cf00b3864db2 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:10.894333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6eebd-75b5-4570-af54-cf00b3864db2 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:10.894333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6eebd-75b5-4570-af54-cf00b3864db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:10.912086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-4cb6eebd-75b5-4570-af54-cf00b3864db2 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:10.913678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4cb6eebd-75b5-4570-af54-cf00b3864db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:10.919931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:10.982170 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1066/2076] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:10.991401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b3c25-225a-4281-96a4-4218d9d1a654 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:10.991690 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b3c25-225a-4281-96a4-4218d9d1a654 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:10.991971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b3c25-225a-4281-96a4-4218d9d1a654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:11.010834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-421b3c25-225a-4281-96a4-4218d9d1a654 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:11.012093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-421b3c25-225a-4281-96a4-4218d9d1a654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:11.021277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1011/2077] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:11.025809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6eebd-75b5-4570-af54-cf00b3864db2 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:11.037716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6eebd-75b5-4570-af54-cf00b3864db2 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:11.042405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081f29c-4e62-4bd9-b2e3-b63a97fb9570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:11.043076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081f29c-4e62-4bd9-b2e3-b63a97fb9570 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:11.043360 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081f29c-4e62-4bd9-b2e3-b63a97fb9570 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:11.043636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081f29c-4e62-4bd9-b2e3-b63a97fb9570 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:11.044373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081f29c-4e62-4bd9-b2e3-b63a97fb9570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:11.044681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081f29c-4e62-4bd9-b2e3-b63a97fb9570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:11.046309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081f29c-4e62-4bd9-b2e3-b63a97fb9570 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:11.058497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1067/2078] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:50:11.062953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b3c25-225a-4281-96a4-4218d9d1a654 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:11.072507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b3c25-225a-4281-96a4-4218d9d1a654 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:11.077604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821a8ba-c6be-42f6-8778-9ddd09f5e559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:11.078265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821a8ba-c6be-42f6-8778-9ddd09f5e559 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:11.078547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821a8ba-c6be-42f6-8778-9ddd09f5e559 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:11.079359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821a8ba-c6be-42f6-8778-9ddd09f5e559 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:11.080054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821a8ba-c6be-42f6-8778-9ddd09f5e559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:11.082004 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821a8ba-c6be-42f6-8778-9ddd09f5e559 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:11.110821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1012/2079] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:11 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:11.115256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081f29c-4e62-4bd9-b2e3-b63a97fb9570 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:11.125814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081f29c-4e62-4bd9-b2e3-b63a97fb9570 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:11.192945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5062853-05b2-43fb-93b9-e9f6a9ff2a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:11.193701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5062853-05b2-43fb-93b9-e9f6a9ff2a2f admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:11.193961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5062853-05b2-43fb-93b9-e9f6a9ff2a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:11.194241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5062853-05b2-43fb-93b9-e9f6a9ff2a2f admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:11.194943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5062853-05b2-43fb-93b9-e9f6a9ff2a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:11.196863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5062853-05b2-43fb-93b9-e9f6a9ff2a2f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:11.222029 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1068/2080] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:11 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:50:11.226234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821a8ba-c6be-42f6-8778-9ddd09f5e559 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:11.237013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821a8ba-c6be-42f6-8778-9ddd09f5e559 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:11.242283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22206d4e-a9d7-4a82-b2d5-bda4985afbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:11.243027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22206d4e-a9d7-4a82-b2d5-bda4985afbfd admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:11.243318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22206d4e-a9d7-4a82-b2d5-bda4985afbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:11.243601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22206d4e-a9d7-4a82-b2d5-bda4985afbfd admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:11.244339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22206d4e-a9d7-4a82-b2d5-bda4985afbfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:11.246306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22206d4e-a9d7-4a82-b2d5-bda4985afbfd admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:11.257937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1013/2081] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:11.262318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5062853-05b2-43fb-93b9-e9f6a9ff2a2f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:11.275445 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5062853-05b2-43fb-93b9-e9f6a9ff2a2f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:11.280241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351dc001-9884-4fe4-9eb1-7748ccae0087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:11.280904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dc001-9884-4fe4-9eb1-7748ccae0087 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:11.281191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dc001-9884-4fe4-9eb1-7748ccae0087 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:11.281510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dc001-9884-4fe4-9eb1-7748ccae0087 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:11.282882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351dc001-9884-4fe4-9eb1-7748ccae0087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:11.284488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351dc001-9884-4fe4-9eb1-7748ccae0087 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:11.297395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1069/2082] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:11.299698 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22206d4e-a9d7-4a82-b2d5-bda4985afbfd admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:11.311776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22206d4e-a9d7-4a82-b2d5-bda4985afbfd admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:11.319546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5a188-40bb-4969-8634-2e924133c7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:11.320457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5a188-40bb-4969-8634-2e924133c7c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:11.320747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5a188-40bb-4969-8634-2e924133c7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:11.321122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5a188-40bb-4969-8634-2e924133c7c5 admin admin] PATH_INFO: `/v3/projects/29a23eb116194101b7577fa0692864d9/users/fde425631fc14e7e9d9bbcadd5e31dc7/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:11.340405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5a188-40bb-4969-8634-2e924133c7c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a23eb116194101b7577fa0692864d9, user_id=fde425631fc14e7e9d9bbcadd5e31dc7, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:11.342821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5a188-40bb-4969-8634-2e924133c7c5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:11.352402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:11.360773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1014/2083] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:11 2022] PUT /identity/v3/projects/29a23eb116194101b7577fa0692864d9/users/fde425631fc14e7e9d9bbcadd5e31dc7/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:11.367066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221eab2-5f22-4535-a7e2-149a1fb8811a None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:11.374150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221eab2-5f22-4535-a7e2-149a1fb8811a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:11.374150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221eab2-5f22-4535-a7e2-149a1fb8811a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:11.406480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-5221eab2-5f22-4535-a7e2-149a1fb8811a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:11.407796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5221eab2-5f22-4535-a7e2-149a1fb8811a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:11.492622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1070/2084] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:11.503937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5a188-40bb-4969-8634-2e924133c7c5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:11.505943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b7a5a188-40bb-4969-8634-2e924133c7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:11.586916 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5a188-40bb-4969-8634-2e924133c7c5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:11.590638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98d163-c5af-4688-9bff-532a990cb353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:11.591290 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d163-c5af-4688-9bff-532a990cb353 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:11.591573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d163-c5af-4688-9bff-532a990cb353 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:11.591846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d163-c5af-4688-9bff-532a990cb353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:11.601873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98d163-c5af-4688-9bff-532a990cb353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:11.603331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98d163-c5af-4688-9bff-532a990cb353 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:11.617303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1015/2085] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:12.506196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8aa561-1c1a-4a55-9e45-0ff813ca80d0 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:12.506735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8aa561-1c1a-4a55-9e45-0ff813ca80d0 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:12.506854 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8aa561-1c1a-4a55-9e45-0ff813ca80d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:12.536136 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-fe8aa561-1c1a-4a55-9e45-0ff813ca80d0 None None] MFA Rules not processed for user `fde425631fc14e7e9d9bbcadd5e31dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:12.537392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fe8aa561-1c1a-4a55-9e45-0ff813ca80d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:12.542489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:12.651363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1071/2086] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:12.656350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98d163-c5af-4688-9bff-532a990cb353 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:12.680237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98d163-c5af-4688-9bff-532a990cb353 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:12.701255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f97a9-0aa6-498a-bdd6-71b10746b777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:12.701963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f97a9-0aa6-498a-bdd6-71b10746b777 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:12.702249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f97a9-0aa6-498a-bdd6-71b10746b777 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:12.702523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f97a9-0aa6-498a-bdd6-71b10746b777 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:12.703226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f97a9-0aa6-498a-bdd6-71b10746b777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:12.705686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f97a9-0aa6-498a-bdd6-71b10746b777 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:12.757145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1016/2087] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:12 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:12.765677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8aa561-1c1a-4a55-9e45-0ff813ca80d0 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:12.815181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8aa561-1c1a-4a55-9e45-0ff813ca80d0 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:12.928073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbdd562-be30-4373-ac22-92b71287132f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:12.928759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdd562-be30-4373-ac22-92b71287132f admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:12.929058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdd562-be30-4373-ac22-92b71287132f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:12.929333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdd562-be30-4373-ac22-92b71287132f admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:12.930059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbdd562-be30-4373-ac22-92b71287132f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:12.932191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbdd562-be30-4373-ac22-92b71287132f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:12.956903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1072/2088] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:12 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:12.962758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f97a9-0aa6-498a-bdd6-71b10746b777 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:12.976207 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f97a9-0aa6-498a-bdd6-71b10746b777 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:12.986681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85f274-2772-458f-b689-1f8c7edd823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:12.988006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85f274-2772-458f-b689-1f8c7edd823b admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:12.988546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85f274-2772-458f-b689-1f8c7edd823b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:12.989061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85f274-2772-458f-b689-1f8c7edd823b admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:12.990129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85f274-2772-458f-b689-1f8c7edd823b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:12.992196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85f274-2772-458f-b689-1f8c7edd823b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:13.011915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1017/2089] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:13.016076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbdd562-be30-4373-ac22-92b71287132f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:13.028646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbdd562-be30-4373-ac22-92b71287132f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:13.035463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66301a4-291d-4a1d-8f7f-f012aa9e4d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:13.036443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66301a4-291d-4a1d-8f7f-f012aa9e4d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:13.036788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66301a4-291d-4a1d-8f7f-f012aa9e4d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:13.037138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66301a4-291d-4a1d-8f7f-f012aa9e4d85 admin admin] PATH_INFO: `/v3/projects/cc46f8938bd3497cb765277a941475f2/users/162e45c1d1f74988ba322d4be5f6b97b/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:13.055678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66301a4-291d-4a1d-8f7f-f012aa9e4d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc46f8938bd3497cb765277a941475f2, user_id=162e45c1d1f74988ba322d4be5f6b97b, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:13.058617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66301a4-291d-4a1d-8f7f-f012aa9e4d85 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:13.077328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1073/2090] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:13 2022] PUT /identity/v3/projects/cc46f8938bd3497cb765277a941475f2/users/162e45c1d1f74988ba322d4be5f6b97b/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:13.081654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85f274-2772-458f-b689-1f8c7edd823b admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:13.092642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85f274-2772-458f-b689-1f8c7edd823b admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:13.174444 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f063a2d-14d2-4aa8-9b5d-f9ce44c43c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:13.175300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f063a2d-14d2-4aa8-9b5d-f9ce44c43c8d admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:13.175675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f063a2d-14d2-4aa8-9b5d-f9ce44c43c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:13.175950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f063a2d-14d2-4aa8-9b5d-f9ce44c43c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:13.176700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f063a2d-14d2-4aa8-9b5d-f9ce44c43c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:13.178364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f063a2d-14d2-4aa8-9b5d-f9ce44c43c8d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:13.189885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1018/2091] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:13.194217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66301a4-291d-4a1d-8f7f-f012aa9e4d85 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:13.206719 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66301a4-291d-4a1d-8f7f-f012aa9e4d85 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:13.211552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e4a34-27f5-4f67-ab75-2dfb4dc6acdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:13.212918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e4a34-27f5-4f67-ab75-2dfb4dc6acdb admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:13.213246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e4a34-27f5-4f67-ab75-2dfb4dc6acdb admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:13.213670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e4a34-27f5-4f67-ab75-2dfb4dc6acdb admin admin] PATH_INFO: `/v3/projects/cc46f8938bd3497cb765277a941475f2/users/162e45c1d1f74988ba322d4be5f6b97b/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:13.216497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e4a34-27f5-4f67-ab75-2dfb4dc6acdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc46f8938bd3497cb765277a941475f2, user_id=162e45c1d1f74988ba322d4be5f6b97b, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:13.222136 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e4a34-27f5-4f67-ab75-2dfb4dc6acdb admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:13.238293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1074/2092] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:13 2022] PUT /identity/v3/projects/cc46f8938bd3497cb765277a941475f2/users/162e45c1d1f74988ba322d4be5f6b97b/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:14.301625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0157f-632e-46bc-865d-51f853bf97f6 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:14.302206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0157f-632e-46bc-865d-51f853bf97f6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:14.302523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0157f-632e-46bc-865d-51f853bf97f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:14.347197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-fcf0157f-632e-46bc-865d-51f853bf97f6 None None] MFA Rules not processed for user `162e45c1d1f74988ba322d4be5f6b97b`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:14.348495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-fcf0157f-632e-46bc-865d-51f853bf97f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:14.477846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:14.544356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e4a34-27f5-4f67-ab75-2dfb4dc6acdb admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:14.545179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1019/2093] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:14.554071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-309e4a34-27f5-4f67-ab75-2dfb4dc6acdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:14.560422 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0157f-632e-46bc-865d-51f853bf97f6 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:14.562206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-fcf0157f-632e-46bc-865d-51f853bf97f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:14.666754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0157f-632e-46bc-865d-51f853bf97f6 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:14.670653 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48e700-2472-44f5-9985-7f0ddf83bd91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:14.671331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48e700-2472-44f5-9985-7f0ddf83bd91 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:14.671640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48e700-2472-44f5-9985-7f0ddf83bd91 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:14.671925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48e700-2472-44f5-9985-7f0ddf83bd91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:14.684559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e4a34-27f5-4f67-ab75-2dfb4dc6acdb admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:14.691976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:14.692936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:14.693340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:14.693806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] PATH_INFO: `/v3/users/111ca1623b6046549ee7e93802ab7ed2` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:14.695418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=111ca1623b6046549ee7e93802ab7ed2)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:14.697791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:14.700208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:14.775039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '111ca1623b6046549ee7e93802ab7ed2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:14.789525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48e700-2472-44f5-9985-7f0ddf83bd91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:14.791929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48e700-2472-44f5-9985-7f0ddf83bd91 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:14.797070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '111ca1623b6046549ee7e93802ab7ed2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:14.809656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1020/2094] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:50:14.819618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '111ca1623b6046549ee7e93802ab7ed2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:14.820494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '111ca1623b6046549ee7e93802ab7ed2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:14.839586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '111ca1623b6046549ee7e93802ab7ed2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:14.858751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1075/2095] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:14 2022] DELETE /identity/v3/users/111ca1623b6046549ee7e93802ab7ed2 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:14.861176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eefb50-a6c4-4c85-a5d0-1a16ceef1757 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:14.861857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eefb50-a6c4-4c85-a5d0-1a16ceef1757 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:14.862245 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eefb50-a6c4-4c85-a5d0-1a16ceef1757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:14.894751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-02eefb50-a6c4-4c85-a5d0-1a16ceef1757 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:14.897053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-02eefb50-a6c4-4c85-a5d0-1a16ceef1757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:14.991431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1021/2096] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:15.006095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:15.008877 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:15.103439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34260089-7439-4ab0-b028-973b9f3ec8cd admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:15.107366 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fa391-a5d0-4474-a0ae-ce43b354a3f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:15.108037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fa391-a5d0-4474-a0ae-ce43b354a3f4 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:15.108322 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fa391-a5d0-4474-a0ae-ce43b354a3f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:15.108605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fa391-a5d0-4474-a0ae-ce43b354a3f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:15.130955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fa391-a5d0-4474-a0ae-ce43b354a3f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:15.133071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fa391-a5d0-4474-a0ae-ce43b354a3f4 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:15.158371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1076/2097] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:15.773226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eefb50-a6c4-4c85-a5d0-1a16ceef1757 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:15.775090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-02eefb50-a6c4-4c85-a5d0-1a16ceef1757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:15.789263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eefb50-a6c4-4c85-a5d0-1a16ceef1757 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:15.794745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8373d1-ae7e-413b-9922-1e0bd57f6dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:15.796074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8373d1-ae7e-413b-9922-1e0bd57f6dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:15.796532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8373d1-ae7e-413b-9922-1e0bd57f6dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:15.797089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8373d1-ae7e-413b-9922-1e0bd57f6dbe admin admin] PATH_INFO: `/v3/users/3dec8074e5f04e5caa18901d1ce0bc66` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:15.800412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8373d1-ae7e-413b-9922-1e0bd57f6dbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dec8074e5f04e5caa18901d1ce0bc66)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:15.802709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8373d1-ae7e-413b-9922-1e0bd57f6dbe admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:15.876869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:15.886245 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-dd8373d1-ae7e-413b-9922-1e0bd57f6dbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dec8074e5f04e5caa18901d1ce0bc66'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:15.897695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-dd8373d1-ae7e-413b-9922-1e0bd57f6dbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dec8074e5f04e5caa18901d1ce0bc66'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:15.903189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fa391-a5d0-4474-a0ae-ce43b354a3f4 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:15.905196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-185fa391-a5d0-4474-a0ae-ce43b354a3f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:15.912182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-dd8373d1-ae7e-413b-9922-1e0bd57f6dbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dec8074e5f04e5caa18901d1ce0bc66'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:15.926757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fa391-a5d0-4474-a0ae-ce43b354a3f4 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:15.930885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-dd8373d1-ae7e-413b-9922-1e0bd57f6dbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dec8074e5f04e5caa18901d1ce0bc66'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:15.931662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-dd8373d1-ae7e-413b-9922-1e0bd57f6dbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dec8074e5f04e5caa18901d1ce0bc66'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:15.937322 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:15.944433 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1022/2098] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:15 2022] DELETE /identity/v3/users/3dec8074e5f04e5caa18901d1ce0bc66 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:15.955190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b3788-bb79-4bcd-a1a3-1f4f73c9c685 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:15.955738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b3788-bb79-4bcd-a1a3-1f4f73c9c685 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:15.956158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:15.956517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b3788-bb79-4bcd-a1a3-1f4f73c9c685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:15.956933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:15.957320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] PATH_INFO: `/v3/projects/6912a800467c48559b89133c81b104ba` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:15.961408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=6912a800467c48559b89133c81b104ba)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:15.963921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:16.003516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-2b1b3788-bb79-4bcd-a1a3-1f4f73c9c685 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:16.004883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2b1b3788-bb79-4bcd-a1a3-1f4f73c9c685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:16.059151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6912a800467c48559b89133c81b104ba', 'request_id': 'req-a84ce21f-fbee-4604-8332-772134fe40ec'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:16.070028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6912a800467c48559b89133c81b104ba', 'request_id': 'req-a84ce21f-fbee-4604-8332-772134fe40ec'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:16.078313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6912a800467c48559b89133c81b104ba', 'request_id': 'req-a84ce21f-fbee-4604-8332-772134fe40ec'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:16.100311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] The token cache is being invalidate because project 6912a800467c48559b89133c81b104ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:16.100879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:16.101886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a84ce21f-fbee-4604-8332-772134fe40ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:16.105096 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1077/2099] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:15 2022] DELETE /identity/v3/projects/6912a800467c48559b89133c81b104ba => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:16.105259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1023/2100] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:16.115232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b748a35-8d1d-4f2b-b721-73f6efd8225b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.115531 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b748a35-8d1d-4f2b-b721-73f6efd8225b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.115820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b748a35-8d1d-4f2b-b721-73f6efd8225b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.120366 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b3788-bb79-4bcd-a1a3-1f4f73c9c685 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:16.122093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2b1b3788-bb79-4bcd-a1a3-1f4f73c9c685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:16.123663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:16.146654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-8b748a35-8d1d-4f2b-b721-73f6efd8225b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:16.148708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8b748a35-8d1d-4f2b-b721-73f6efd8225b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:16.224262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b3788-bb79-4bcd-a1a3-1f4f73c9c685 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:16.264510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1078/2101] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:16.272769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba1a8b-e4e3-4f50-99d0-ed302bf834d7 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.273057 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba1a8b-e4e3-4f50-99d0-ed302bf834d7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.273344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba1a8b-e4e3-4f50-99d0-ed302bf834d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.292440 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aafa0-10a8-4313-bc16-104a0929391d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:16.293275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aafa0-10a8-4313-bc16-104a0929391d service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.293697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aafa0-10a8-4313-bc16-104a0929391d service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.294108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aafa0-10a8-4313-bc16-104a0929391d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.296098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b59aafa0-10a8-4313-bc16-104a0929391d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:16.298447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f8ba1a8b-e4e3-4f50-99d0-ed302bf834d7 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:16.299663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f8ba1a8b-e4e3-4f50-99d0-ed302bf834d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:16.308778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1079/2102] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:16.310887 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59aafa0-10a8-4313-bc16-104a0929391d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:16.312445 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59aafa0-10a8-4313-bc16-104a0929391d service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:16.312964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba1a8b-e4e3-4f50-99d0-ed302bf834d7 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:16.326680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba1a8b-e4e3-4f50-99d0-ed302bf834d7 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:16.328173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1024/2103] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:16.331977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca73df9-92fb-4f3a-8528-dd4732a8ba6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:16.333158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca73df9-92fb-4f3a-8528-dd4732a8ba6e admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.333480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca73df9-92fb-4f3a-8528-dd4732a8ba6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.333762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca73df9-92fb-4f3a-8528-dd4732a8ba6e admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.334510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca73df9-92fb-4f3a-8528-dd4732a8ba6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:16.334824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca73df9-92fb-4f3a-8528-dd4732a8ba6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:16.336788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca73df9-92fb-4f3a-8528-dd4732a8ba6e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:16.351576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1080/2104] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:50:16.356128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aafa0-10a8-4313-bc16-104a0929391d service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:16.369204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aafa0-10a8-4313-bc16-104a0929391d service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:16.377333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d7223-aa08-4e65-a803-384ac74933af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:16.377537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d7223-aa08-4e65-a803-384ac74933af admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.377828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d7223-aa08-4e65-a803-384ac74933af admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.378194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d7223-aa08-4e65-a803-384ac74933af admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.379517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d7223-aa08-4e65-a803-384ac74933af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:16.381824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d7223-aa08-4e65-a803-384ac74933af admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:16.403134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:16.427515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1025/2105] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:16 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:50:16.439505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca73df9-92fb-4f3a-8528-dd4732a8ba6e admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:16.450188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d7223-aa08-4e65-a803-384ac74933af admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:16.452309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7e1d7223-aa08-4e65-a803-384ac74933af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:16.452505 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca73df9-92fb-4f3a-8528-dd4732a8ba6e admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:16.532864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724c1ad-7e21-4c55-ab79-ab7cd46fd874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:16.536407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724c1ad-7e21-4c55-ab79-ab7cd46fd874 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.536945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724c1ad-7e21-4c55-ab79-ab7cd46fd874 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.537572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724c1ad-7e21-4c55-ab79-ab7cd46fd874 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.539716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724c1ad-7e21-4c55-ab79-ab7cd46fd874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:16.542984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724c1ad-7e21-4c55-ab79-ab7cd46fd874 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:16.553399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d7223-aa08-4e65-a803-384ac74933af admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:16.565995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812bc86-bc61-482a-9a9c-3def39ac9291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:16.566673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812bc86-bc61-482a-9a9c-3def39ac9291 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.566966 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812bc86-bc61-482a-9a9c-3def39ac9291 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.567252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812bc86-bc61-482a-9a9c-3def39ac9291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.570990 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e812bc86-bc61-482a-9a9c-3def39ac9291 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:16.590849 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1081/2106] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:16 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:16.596067 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724c1ad-7e21-4c55-ab79-ab7cd46fd874 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:16.614044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724c1ad-7e21-4c55-ab79-ab7cd46fd874 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:16.622755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb4957-8564-48f0-befb-e17280432771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:16.624495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb4957-8564-48f0-befb-e17280432771 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.624788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb4957-8564-48f0-befb-e17280432771 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.625072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb4957-8564-48f0-befb-e17280432771 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.626675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb4957-8564-48f0-befb-e17280432771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:16.628360 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb4957-8564-48f0-befb-e17280432771 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:16.656269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1082/2107] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:16.658997 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb4957-8564-48f0-befb-e17280432771 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:16.681035 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb4957-8564-48f0-befb-e17280432771 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:16.687733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea6f79-cb67-4f0a-85d8-303cc33f7343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:16.688459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea6f79-cb67-4f0a-85d8-303cc33f7343 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.688753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea6f79-cb67-4f0a-85d8-303cc33f7343 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.689036 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea6f79-cb67-4f0a-85d8-303cc33f7343 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.690430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea6f79-cb67-4f0a-85d8-303cc33f7343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:16.691456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea6f79-cb67-4f0a-85d8-303cc33f7343 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:16.716585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812bc86-bc61-482a-9a9c-3def39ac9291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:16.718093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812bc86-bc61-482a-9a9c-3def39ac9291 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:16.724390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1083/2108] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:16.725617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea6f79-cb67-4f0a-85d8-303cc33f7343 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:16.732490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1026/2109] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:50:16.745772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea6f79-cb67-4f0a-85d8-303cc33f7343 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:16.763088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcbe28-ae8c-4a58-964e-a0df3771615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:16.764085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcbe28-ae8c-4a58-964e-a0df3771615a admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.764275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcbe28-ae8c-4a58-964e-a0df3771615a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.764702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcbe28-ae8c-4a58-964e-a0df3771615a admin admin] PATH_INFO: `/v3/projects/cea30f16a8ab481794d8283d76fd65c9/users/89f176eb3711427d93a2dabc2fd7d093/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.791059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812bc86-bc61-482a-9a9c-3def39ac9291 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:16.793268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e812bc86-bc61-482a-9a9c-3def39ac9291 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:16.840512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcbe28-ae8c-4a58-964e-a0df3771615a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea30f16a8ab481794d8283d76fd65c9, user_id=89f176eb3711427d93a2dabc2fd7d093, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:16.849939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcbe28-ae8c-4a58-964e-a0df3771615a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:16.861965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812bc86-bc61-482a-9a9c-3def39ac9291 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:16.869093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:16.870405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.870704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.871026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] PATH_INFO: `/v3/projects/4091c0c016c8466cadf3bcf92cb21030` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.872606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4091c0c016c8466cadf3bcf92cb21030)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:16.874712 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:16.877509 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1084/2110] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:16 2022] PUT /identity/v3/projects/cea30f16a8ab481794d8283d76fd65c9/users/89f176eb3711427d93a2dabc2fd7d093/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:16.889794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec72060-4a67-483d-9397-3087fc3685f6 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:16.890846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec72060-4a67-483d-9397-3087fc3685f6 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:16.892416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec72060-4a67-483d-9397-3087fc3685f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:16.929752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-7ec72060-4a67-483d-9397-3087fc3685f6 None None] MFA Rules not processed for user `89f176eb3711427d93a2dabc2fd7d093`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:16.932201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-7ec72060-4a67-483d-9397-3087fc3685f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:16.957342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4091c0c016c8466cadf3bcf92cb21030', 'request_id': 'req-acca9b1a-3b47-4915-b261-475be0bfb828'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:16.964212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:16.971199 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4091c0c016c8466cadf3bcf92cb21030', 'request_id': 'req-acca9b1a-3b47-4915-b261-475be0bfb828'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:16.978717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4091c0c016c8466cadf3bcf92cb21030', 'request_id': 'req-acca9b1a-3b47-4915-b261-475be0bfb828'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.014462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] The token cache is being invalidate because project 4091c0c016c8466cadf3bcf92cb21030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:17.014765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.015459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.018187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1027/2111] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:16 2022] DELETE /identity/v3/projects/4091c0c016c8466cadf3bcf92cb21030 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:17.021913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:17.024103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:17.076283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1085/2112] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:17.089594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec72060-4a67-483d-9397-3087fc3685f6 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:17.090983 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-7ec72060-4a67-483d-9397-3087fc3685f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:17.113550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:17.166947 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca9b1a-3b47-4915-b261-475be0bfb828 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:17.176401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:17.177581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:17.178575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:17.179432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] PATH_INFO: `/v3/users/fd8d23c5a1ff406eaa56444832e29c2b` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:17.181866 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd8d23c5a1ff406eaa56444832e29c2b)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:17.183902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:17.207207 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec72060-4a67-483d-9397-3087fc3685f6 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:17.265036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd8d23c5a1ff406eaa56444832e29c2b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.279276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd8d23c5a1ff406eaa56444832e29c2b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.314682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd8d23c5a1ff406eaa56444832e29c2b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.342492 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd8d23c5a1ff406eaa56444832e29c2b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.343759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd8d23c5a1ff406eaa56444832e29c2b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.343885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c670fe-2fb9-4c5b-a259-d475f50435d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:17.345689 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c670fe-2fb9-4c5b-a259-d475f50435d7 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:17.346734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c670fe-2fb9-4c5b-a259-d475f50435d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:17.347177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c670fe-2fb9-4c5b-a259-d475f50435d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:17.350311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c9c670fe-2fb9-4c5b-a259-d475f50435d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:17.357412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1028/2113] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:17 2022] DELETE /identity/v3/users/fd8d23c5a1ff406eaa56444832e29c2b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:17.468085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c670fe-2fb9-4c5b-a259-d475f50435d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:17.471169 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c670fe-2fb9-4c5b-a259-d475f50435d7 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:17.482044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1086/2114] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:17.707241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:17.708780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:17.800453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e169cb-ae3b-477f-b02f-7f9205d8c6cd admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:17.806262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:17.807661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:17.808051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:17.808432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] PATH_INFO: `/v3/projects/8507215fa9564a1c8365c782a146527a` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:17.810530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8507215fa9564a1c8365c782a146527a)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:17.812812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:17.868778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:17.870890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8507215fa9564a1c8365c782a146527a', 'request_id': 'req-8656d432-fac9-47e6-b46d-00e57b3d44fc'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.880212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8507215fa9564a1c8365c782a146527a', 'request_id': 'req-8656d432-fac9-47e6-b46d-00e57b3d44fc'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.886976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8507215fa9564a1c8365c782a146527a', 'request_id': 'req-8656d432-fac9-47e6-b46d-00e57b3d44fc'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.901385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] The token cache is being invalidate because project 8507215fa9564a1c8365c782a146527a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:17.901702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.902401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8656d432-fac9-47e6-b46d-00e57b3d44fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:17.903959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1029/2115] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:17 2022] DELETE /identity/v3/projects/8507215fa9564a1c8365c782a146527a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:17.908881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06956c47-f77e-431f-afa7-86660711fb16 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:17.909173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06956c47-f77e-431f-afa7-86660711fb16 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:17.909488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06956c47-f77e-431f-afa7-86660711fb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:17.928632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-06956c47-f77e-431f-afa7-86660711fb16 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:17.929964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-06956c47-f77e-431f-afa7-86660711fb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:17.961996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:18.001968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1087/2116] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:18.009867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52cd39-6a37-4078-bc85-a04ecd87bcf2 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.010158 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52cd39-6a37-4078-bc85-a04ecd87bcf2 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.010441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52cd39-6a37-4078-bc85-a04ecd87bcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.031121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-db52cd39-6a37-4078-bc85-a04ecd87bcf2 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:18.032370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-db52cd39-6a37-4078-bc85-a04ecd87bcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:18.041036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1030/2117] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:18.045122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06956c47-f77e-431f-afa7-86660711fb16 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.056390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06956c47-f77e-431f-afa7-86660711fb16 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.061477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118696-32bd-4846-8f5f-42e516d6cacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.062156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118696-32bd-4846-8f5f-42e516d6cacc admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.062448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118696-32bd-4846-8f5f-42e516d6cacc admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.062724 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118696-32bd-4846-8f5f-42e516d6cacc admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.064029 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f118696-32bd-4846-8f5f-42e516d6cacc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:18.064345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f118696-32bd-4846-8f5f-42e516d6cacc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.065991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f118696-32bd-4846-8f5f-42e516d6cacc admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.083636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1088/2118] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:50:18.083986 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52cd39-6a37-4078-bc85-a04ecd87bcf2 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.094865 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52cd39-6a37-4078-bc85-a04ecd87bcf2 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.099637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fefa5c-a839-4841-9d48-a4857479fd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.100300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fefa5c-a839-4841-9d48-a4857479fd69 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.100583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fefa5c-a839-4841-9d48-a4857479fd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.100861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fefa5c-a839-4841-9d48-a4857479fd69 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.101640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fefa5c-a839-4841-9d48-a4857479fd69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.103531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fefa5c-a839-4841-9d48-a4857479fd69 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.131451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1031/2119] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:18 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:18.135408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118696-32bd-4846-8f5f-42e516d6cacc admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.145560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118696-32bd-4846-8f5f-42e516d6cacc admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.208104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fff6e5-0144-4559-8dfb-e56daede23b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.208363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fff6e5-0144-4559-8dfb-e56daede23b6 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.208646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fff6e5-0144-4559-8dfb-e56daede23b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.208946 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fff6e5-0144-4559-8dfb-e56daede23b6 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.209730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fff6e5-0144-4559-8dfb-e56daede23b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.212074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fff6e5-0144-4559-8dfb-e56daede23b6 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.234564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1089/2120] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:18 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:18.239572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fefa5c-a839-4841-9d48-a4857479fd69 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.250758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fefa5c-a839-4841-9d48-a4857479fd69 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.255691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0100e07-0381-4eda-9b44-5445ae3049b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.256483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100e07-0381-4eda-9b44-5445ae3049b9 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.256852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100e07-0381-4eda-9b44-5445ae3049b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.257222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100e07-0381-4eda-9b44-5445ae3049b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.258077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0100e07-0381-4eda-9b44-5445ae3049b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.259969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0100e07-0381-4eda-9b44-5445ae3049b9 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.275994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1032/2121] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:18.277172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fff6e5-0144-4559-8dfb-e56daede23b6 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.287939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fff6e5-0144-4559-8dfb-e56daede23b6 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.294264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e134eb5-2998-4e27-ab18-26ae87d5fddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.294955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e134eb5-2998-4e27-ab18-26ae87d5fddf admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.295238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e134eb5-2998-4e27-ab18-26ae87d5fddf admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.295513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e134eb5-2998-4e27-ab18-26ae87d5fddf admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.296241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e134eb5-2998-4e27-ab18-26ae87d5fddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.298428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e134eb5-2998-4e27-ab18-26ae87d5fddf admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.310609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1090/2122] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:18.315034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0100e07-0381-4eda-9b44-5445ae3049b9 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.325420 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0100e07-0381-4eda-9b44-5445ae3049b9 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.330397 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51119b7e-696b-447f-9afd-12506aafc950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.331558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51119b7e-696b-447f-9afd-12506aafc950 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.331982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51119b7e-696b-447f-9afd-12506aafc950 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.332274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51119b7e-696b-447f-9afd-12506aafc950 admin admin] PATH_INFO: `/v3/projects/50cbe058356a4b768c50bcffc605c7e4/users/024779d6e6c74835a43d4e68ded23281/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.349130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51119b7e-696b-447f-9afd-12506aafc950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50cbe058356a4b768c50bcffc605c7e4, user_id=024779d6e6c74835a43d4e68ded23281, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.351937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51119b7e-696b-447f-9afd-12506aafc950 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.371281 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1033/2123] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:18 2022] PUT /identity/v3/projects/50cbe058356a4b768c50bcffc605c7e4/users/024779d6e6c74835a43d4e68ded23281/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:18.382022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ec098-f9c6-4559-984c-5513e255bf8c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.382299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ec098-f9c6-4559-984c-5513e255bf8c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.382583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ec098-f9c6-4559-984c-5513e255bf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.420172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-501ec098-f9c6-4559-984c-5513e255bf8c None None] MFA Rules not processed for user `024779d6e6c74835a43d4e68ded23281`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:18.421824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-501ec098-f9c6-4559-984c-5513e255bf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:18.562318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1091/2124] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:18.567176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51119b7e-696b-447f-9afd-12506aafc950 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.666971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51119b7e-696b-447f-9afd-12506aafc950 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.672991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8d12c-597e-488a-a5e7-a4c6e9ba3517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.674049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8d12c-597e-488a-a5e7-a4c6e9ba3517 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.674443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8d12c-597e-488a-a5e7-a4c6e9ba3517 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.674796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8d12c-597e-488a-a5e7-a4c6e9ba3517 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.675595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8d12c-597e-488a-a5e7-a4c6e9ba3517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.677709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8d12c-597e-488a-a5e7-a4c6e9ba3517 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.721248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1034/2125] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:18 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:18.728389 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ec098-f9c6-4559-984c-5513e255bf8c None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.745707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ec098-f9c6-4559-984c-5513e255bf8c None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.775408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:18.837874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e49e4-d236-4920-b0e3-f579af24f001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.838580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e49e4-d236-4920-b0e3-f579af24f001 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.838865 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e49e4-d236-4920-b0e3-f579af24f001 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.839141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e49e4-d236-4920-b0e3-f579af24f001 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.839854 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e49e4-d236-4920-b0e3-f579af24f001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.841915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e49e4-d236-4920-b0e3-f579af24f001 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.890569 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1092/2126] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:18 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:18.892443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8d12c-597e-488a-a5e7-a4c6e9ba3517 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.910658 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8d12c-597e-488a-a5e7-a4c6e9ba3517 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.926391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7c428-2bb3-4af1-9fc3-897e2321ba5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.927447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7c428-2bb3-4af1-9fc3-897e2321ba5e admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.927999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7c428-2bb3-4af1-9fc3-897e2321ba5e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.928190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7c428-2bb3-4af1-9fc3-897e2321ba5e admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.928852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7c428-2bb3-4af1-9fc3-897e2321ba5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.930548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7c428-2bb3-4af1-9fc3-897e2321ba5e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.945586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1035/2127] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:18.950073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e49e4-d236-4920-b0e3-f579af24f001 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.962134 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e49e4-d236-4920-b0e3-f579af24f001 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.967190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455dffa-2c1a-4ac6-9615-56a2dd8bd00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.967874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455dffa-2c1a-4ac6-9615-56a2dd8bd00e admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.968160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455dffa-2c1a-4ac6-9615-56a2dd8bd00e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.968695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455dffa-2c1a-4ac6-9615-56a2dd8bd00e admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.969458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455dffa-2c1a-4ac6-9615-56a2dd8bd00e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.971172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455dffa-2c1a-4ac6-9615-56a2dd8bd00e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.984510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1093/2128] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:50:18.988853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7c428-2bb3-4af1-9fc3-897e2321ba5e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:19.003661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7c428-2bb3-4af1-9fc3-897e2321ba5e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:19.023210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de19b37-d9a5-447a-a3bf-2107dbefa50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:19.024713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de19b37-d9a5-447a-a3bf-2107dbefa50d admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.025198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de19b37-d9a5-447a-a3bf-2107dbefa50d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.025730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de19b37-d9a5-447a-a3bf-2107dbefa50d admin admin] PATH_INFO: `/v3/projects/55ae472d39d341fda42b6e07711a7d74/users/804d65cd3f4a464ea967ea4ee1f50a18/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.049747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de19b37-d9a5-447a-a3bf-2107dbefa50d admin admin] RBAC: Authorizing `identity:create_grant(project_id=55ae472d39d341fda42b6e07711a7d74, user_id=804d65cd3f4a464ea967ea4ee1f50a18, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:19.052667 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de19b37-d9a5-447a-a3bf-2107dbefa50d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:19.070415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1036/2129] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:18 2022] PUT /identity/v3/projects/55ae472d39d341fda42b6e07711a7d74/users/804d65cd3f4a464ea967ea4ee1f50a18/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:19.082892 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbadfa-6368-495e-9167-1b5ec06e7357 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.083511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbadfa-6368-495e-9167-1b5ec06e7357 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.084028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbadfa-6368-495e-9167-1b5ec06e7357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.119128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f7fbadfa-6368-495e-9167-1b5ec06e7357 None None] MFA Rules not processed for user `804d65cd3f4a464ea967ea4ee1f50a18`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:19.120542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f7fbadfa-6368-495e-9167-1b5ec06e7357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:19.232663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1094/2130] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:19.234374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de19b37-d9a5-447a-a3bf-2107dbefa50d admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:19.269201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:19.341664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de19b37-d9a5-447a-a3bf-2107dbefa50d admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:19.350917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25af9a0-b7a6-4d9c-842b-05bc46f713cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:19.352121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25af9a0-b7a6-4d9c-842b-05bc46f713cf admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.352374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25af9a0-b7a6-4d9c-842b-05bc46f713cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.352811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25af9a0-b7a6-4d9c-842b-05bc46f713cf admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.353582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25af9a0-b7a6-4d9c-842b-05bc46f713cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:19.356171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25af9a0-b7a6-4d9c-842b-05bc46f713cf admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:19.406843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1037/2131] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:19 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:19.412350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbadfa-6368-495e-9167-1b5ec06e7357 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:19.429866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbadfa-6368-495e-9167-1b5ec06e7357 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:19.517644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8042e2-d3e7-4a43-953f-cb844173d171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:19.518359 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8042e2-d3e7-4a43-953f-cb844173d171 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.518723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8042e2-d3e7-4a43-953f-cb844173d171 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.519020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8042e2-d3e7-4a43-953f-cb844173d171 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.519715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8042e2-d3e7-4a43-953f-cb844173d171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:19.521823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8042e2-d3e7-4a43-953f-cb844173d171 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:19.555590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1095/2132] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:19 2022] POST /identity/v3/users => generated 295 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:19.556709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25af9a0-b7a6-4d9c-842b-05bc46f713cf admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:19.575374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25af9a0-b7a6-4d9c-842b-05bc46f713cf admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:19.581884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151791af-9fd1-4d6c-88ba-0b96d568265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:19.582670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151791af-9fd1-4d6c-88ba-0b96d568265c admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.583089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151791af-9fd1-4d6c-88ba-0b96d568265c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.583580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151791af-9fd1-4d6c-88ba-0b96d568265c admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.584734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151791af-9fd1-4d6c-88ba-0b96d568265c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:19.587125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151791af-9fd1-4d6c-88ba-0b96d568265c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:19.612357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1038/2133] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:50:19.616336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8042e2-d3e7-4a43-953f-cb844173d171 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:19.639801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8042e2-d3e7-4a43-953f-cb844173d171 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:19.652821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66c070-4c06-4d59-8738-fe7e1619558b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:19.654053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66c070-4c06-4d59-8738-fe7e1619558b admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.654432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66c070-4c06-4d59-8738-fe7e1619558b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.654803 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66c070-4c06-4d59-8738-fe7e1619558b admin admin] PATH_INFO: `/v3/projects/a1f6e185ee414969b562e645000ad48c/users/aabfddc1c5e44ca79ebf119c77375506/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.681834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66c070-4c06-4d59-8738-fe7e1619558b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1f6e185ee414969b562e645000ad48c, user_id=aabfddc1c5e44ca79ebf119c77375506, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:19.684566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66c070-4c06-4d59-8738-fe7e1619558b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:19.697054 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:19.717908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1096/2134] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:19 2022] PUT /identity/v3/projects/a1f6e185ee414969b562e645000ad48c/users/aabfddc1c5e44ca79ebf119c77375506/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:19.718087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151791af-9fd1-4d6c-88ba-0b96d568265c admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:19.734744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151791af-9fd1-4d6c-88ba-0b96d568265c admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:19.821194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4acc3-c1bc-4cf5-acbc-4cff136acdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:19.822732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4acc3-c1bc-4cf5-acbc-4cff136acdfc admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.824027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4acc3-c1bc-4cf5-acbc-4cff136acdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.824449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4acc3-c1bc-4cf5-acbc-4cff136acdfc admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.825464 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4acc3-c1bc-4cf5-acbc-4cff136acdfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:19.831212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4acc3-c1bc-4cf5-acbc-4cff136acdfc admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:19.848899 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1039/2135] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:19.853662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66c070-4c06-4d59-8738-fe7e1619558b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:19.871230 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66c070-4c06-4d59-8738-fe7e1619558b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:19.886409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032348d2-f768-4151-90b6-e6e60d63591e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:19.887406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032348d2-f768-4151-90b6-e6e60d63591e admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.887662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032348d2-f768-4151-90b6-e6e60d63591e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.888057 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032348d2-f768-4151-90b6-e6e60d63591e admin admin] PATH_INFO: `/v3/projects/a1f6e185ee414969b562e645000ad48c/users/aabfddc1c5e44ca79ebf119c77375506/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.892581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032348d2-f768-4151-90b6-e6e60d63591e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1f6e185ee414969b562e645000ad48c, user_id=aabfddc1c5e44ca79ebf119c77375506, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:19.901081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032348d2-f768-4151-90b6-e6e60d63591e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:19.926598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1097/2136] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:19 2022] PUT /identity/v3/projects/a1f6e185ee414969b562e645000ad48c/users/aabfddc1c5e44ca79ebf119c77375506/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:19.937328 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96addae1-3b36-4133-a0c0-98950e36a089 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.937806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96addae1-3b36-4133-a0c0-98950e36a089 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.937943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96addae1-3b36-4133-a0c0-98950e36a089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:20.006392 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-96addae1-3b36-4133-a0c0-98950e36a089 None None] MFA Rules not processed for user `aabfddc1c5e44ca79ebf119c77375506`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:20.011015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-96addae1-3b36-4133-a0c0-98950e36a089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:20.144528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1040/2137] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:20.158825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032348d2-f768-4151-90b6-e6e60d63591e admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:20.162578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-032348d2-f768-4151-90b6-e6e60d63591e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:20.277972 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032348d2-f768-4151-90b6-e6e60d63591e admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:20.282848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3362c92-0774-4c7c-b34d-585b2d99334f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:20.283701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3362c92-0774-4c7c-b34d-585b2d99334f service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:20.284169 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3362c92-0774-4c7c-b34d-585b2d99334f service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:20.284473 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3362c92-0774-4c7c-b34d-585b2d99334f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:20.420348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3362c92-0774-4c7c-b34d-585b2d99334f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:20.422036 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3362c92-0774-4c7c-b34d-585b2d99334f service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:20.441666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1098/2138] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:20.958889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96addae1-3b36-4133-a0c0-98950e36a089 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:20.960548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-96addae1-3b36-4133-a0c0-98950e36a089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:20.986345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:21.045792 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96addae1-3b36-4133-a0c0-98950e36a089 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:21.052123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:21.053553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:21.053553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:21.053553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] PATH_INFO: `/v3/users/89f176eb3711427d93a2dabc2fd7d093` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:21.055187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=89f176eb3711427d93a2dabc2fd7d093)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:21.057117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:21.125579 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89f176eb3711427d93a2dabc2fd7d093'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:21.127295 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89f176eb3711427d93a2dabc2fd7d093'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:21.136432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89f176eb3711427d93a2dabc2fd7d093'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:21.152070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89f176eb3711427d93a2dabc2fd7d093'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:21.152831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89f176eb3711427d93a2dabc2fd7d093'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:21.160076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1041/2139] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:20 2022] DELETE /identity/v3/users/89f176eb3711427d93a2dabc2fd7d093 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:21.164424 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c81c-8130-4091-9b68-038a300d895e None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:21.164909 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c81c-8130-4091-9b68-038a300d895e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:21.165024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c81c-8130-4091-9b68-038a300d895e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:21.185113 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-6486c81c-8130-4091-9b68-038a300d895e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:21.186463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6486c81c-8130-4091-9b68-038a300d895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:21.326452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1099/2140] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:21.349142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:21.351840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:21.485112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dbdcf-abc6-422c-a2cd-05401fb070ba admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:21.496784 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f89a3b-20ad-457b-92d3-b6b6dddb1601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:21.497689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f89a3b-20ad-457b-92d3-b6b6dddb1601 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:21.505745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f89a3b-20ad-457b-92d3-b6b6dddb1601 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:21.506447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f89a3b-20ad-457b-92d3-b6b6dddb1601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:21.519464 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f89a3b-20ad-457b-92d3-b6b6dddb1601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:21.521400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f89a3b-20ad-457b-92d3-b6b6dddb1601 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:21.533833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1042/2141] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:21.949453 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486c81c-8130-4091-9b68-038a300d895e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:21.951720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6486c81c-8130-4091-9b68-038a300d895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:21.974783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486c81c-8130-4091-9b68-038a300d895e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:21.982978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:21.984158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:21.984462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:21.984806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] PATH_INFO: `/v3/projects/cea30f16a8ab481794d8283d76fd65c9` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:21.986455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea30f16a8ab481794d8283d76fd65c9)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:21.989169 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:22.040012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:22.049469 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f89a3b-20ad-457b-92d3-b6b6dddb1601 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:22.051049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a9f89a3b-20ad-457b-92d3-b6b6dddb1601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:22.054951 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea30f16a8ab481794d8283d76fd65c9', 'request_id': 'req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:22.067392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea30f16a8ab481794d8283d76fd65c9', 'request_id': 'req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:22.074493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea30f16a8ab481794d8283d76fd65c9', 'request_id': 'req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:22.088600 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] The token cache is being invalidate because project cea30f16a8ab481794d8283d76fd65c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:22.088999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:22.089798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4ad0ffde-81a6-4b32-b0b5-b5f0b62c62ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:22.091420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1100/2142] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:21 2022] DELETE /identity/v3/projects/cea30f16a8ab481794d8283d76fd65c9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:22.096465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd438-9f93-420e-9a47-0da54ea7f8a7 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:22.097790 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd438-9f93-420e-9a47-0da54ea7f8a7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:22.098136 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd438-9f93-420e-9a47-0da54ea7f8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:22.116793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:22.122147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-314fd438-9f93-420e-9a47-0da54ea7f8a7 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:22.123614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-314fd438-9f93-420e-9a47-0da54ea7f8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:22.135051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f89a3b-20ad-457b-92d3-b6b6dddb1601 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:22.196550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191d0a4-fed3-437f-b4f4-b5d622ea3636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:22.197322 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1101/2143] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:22.198615 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191d0a4-fed3-437f-b4f4-b5d622ea3636 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:22.198909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191d0a4-fed3-437f-b4f4-b5d622ea3636 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:22.199263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191d0a4-fed3-437f-b4f4-b5d622ea3636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:22.204557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b191d0a4-fed3-437f-b4f4-b5d622ea3636 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:22.205430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598cdcd-98a5-42da-b9f9-ba344419165d None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:22.205726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598cdcd-98a5-42da-b9f9-ba344419165d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:22.206008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598cdcd-98a5-42da-b9f9-ba344419165d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:22.227420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-4598cdcd-98a5-42da-b9f9-ba344419165d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:22.229825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4598cdcd-98a5-42da-b9f9-ba344419165d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:22.242238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1102/2144] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:22.246861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598cdcd-98a5-42da-b9f9-ba344419165d None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:22.257508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598cdcd-98a5-42da-b9f9-ba344419165d None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:22.262706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe62fb-f38c-4919-9ab5-36fc279b3743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:22.263378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe62fb-f38c-4919-9ab5-36fc279b3743 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:22.263669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe62fb-f38c-4919-9ab5-36fc279b3743 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:22.264155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe62fb-f38c-4919-9ab5-36fc279b3743 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:22.265172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe62fb-f38c-4919-9ab5-36fc279b3743 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:22.265535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe62fb-f38c-4919-9ab5-36fc279b3743 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:22.267181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe62fb-f38c-4919-9ab5-36fc279b3743 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:22.280400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1103/2145] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:50:22.284873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe62fb-f38c-4919-9ab5-36fc279b3743 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:22.285051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191d0a4-fed3-437f-b4f4-b5d622ea3636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:22.287298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191d0a4-fed3-437f-b4f4-b5d622ea3636 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:22.296071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe62fb-f38c-4919-9ab5-36fc279b3743 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:22.300077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1043/2146] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:22.303567 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2329cc5-50d3-4152-9a5f-87f287d59806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:22.304253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2329cc5-50d3-4152-9a5f-87f287d59806 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:22.304606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2329cc5-50d3-4152-9a5f-87f287d59806 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:22.304886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2329cc5-50d3-4152-9a5f-87f287d59806 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:22.305658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2329cc5-50d3-4152-9a5f-87f287d59806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:22.308220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2329cc5-50d3-4152-9a5f-87f287d59806 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:22.365166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1104/2147] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:22 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:50:22.366701 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191d0a4-fed3-437f-b4f4-b5d622ea3636 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:22.395840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191d0a4-fed3-437f-b4f4-b5d622ea3636 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:22.515735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88dfa-f816-4185-a72b-e8b1d39dc213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:22.516515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88dfa-f816-4185-a72b-e8b1d39dc213 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:22.516805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88dfa-f816-4185-a72b-e8b1d39dc213 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:22.517082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88dfa-f816-4185-a72b-e8b1d39dc213 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:22.523779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d88dfa-f816-4185-a72b-e8b1d39dc213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:22.530782 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d88dfa-f816-4185-a72b-e8b1d39dc213 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:22.588411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1044/2148] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:22 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:22.589816 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2329cc5-50d3-4152-9a5f-87f287d59806 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:22.611245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2329cc5-50d3-4152-9a5f-87f287d59806 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:22.616332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d6acf-2ccf-42db-98ff-cb54c06c45c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:22.625746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d6acf-2ccf-42db-98ff-cb54c06c45c2 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:22.625746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d6acf-2ccf-42db-98ff-cb54c06c45c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:22.625746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d6acf-2ccf-42db-98ff-cb54c06c45c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:22.625746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d6acf-2ccf-42db-98ff-cb54c06c45c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:22.625746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d6acf-2ccf-42db-98ff-cb54c06c45c2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:22.646779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1105/2149] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:22.649575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88dfa-f816-4185-a72b-e8b1d39dc213 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:22.660813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88dfa-f816-4185-a72b-e8b1d39dc213 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:22.665851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756bf72-e86a-434b-b4c7-e4d68245dbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:22.666525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756bf72-e86a-434b-b4c7-e4d68245dbcd admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:22.666991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756bf72-e86a-434b-b4c7-e4d68245dbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:22.667274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756bf72-e86a-434b-b4c7-e4d68245dbcd admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:22.668017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756bf72-e86a-434b-b4c7-e4d68245dbcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:22.669660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756bf72-e86a-434b-b4c7-e4d68245dbcd admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:22.686798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1045/2150] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:22.688055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d6acf-2ccf-42db-98ff-cb54c06c45c2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:22.705164 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d6acf-2ccf-42db-98ff-cb54c06c45c2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:22.723371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7a912-64c5-43cf-ab60-8397c3ae2e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:22.724758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7a912-64c5-43cf-ab60-8397c3ae2e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:22.725296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7a912-64c5-43cf-ab60-8397c3ae2e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:22.725803 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7a912-64c5-43cf-ab60-8397c3ae2e18 admin admin] PATH_INFO: `/v3/projects/fbb36de9e4fb481da5509fd646eff534/users/da7299538b9b4e0186db26e7612bfe16/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:22.760580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7a912-64c5-43cf-ab60-8397c3ae2e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb36de9e4fb481da5509fd646eff534, user_id=da7299538b9b4e0186db26e7612bfe16, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:22.763420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7a912-64c5-43cf-ab60-8397c3ae2e18 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:22.796027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1106/2151] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:22 2022] PUT /identity/v3/projects/fbb36de9e4fb481da5509fd646eff534/users/da7299538b9b4e0186db26e7612bfe16/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:22.808981 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b20fa-b7d6-43b1-a60d-5b98c007f341 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:22.809263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b20fa-b7d6-43b1-a60d-5b98c007f341 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:22.809824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b20fa-b7d6-43b1-a60d-5b98c007f341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:22.852418 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-d42b20fa-b7d6-43b1-a60d-5b98c007f341 None None] MFA Rules not processed for user `da7299538b9b4e0186db26e7612bfe16`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:22.854062 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d42b20fa-b7d6-43b1-a60d-5b98c007f341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:22.908256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:23.056774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1046/2152] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:23.161931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7a912-64c5-43cf-ab60-8397c3ae2e18 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:23.166285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-37e7a912-64c5-43cf-ab60-8397c3ae2e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:23.264935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:23.275307 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7a912-64c5-43cf-ab60-8397c3ae2e18 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:23.282033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5155c-5b44-4c1e-a5bd-1b9a15254cce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:23.283207 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5155c-5b44-4c1e-a5bd-1b9a15254cce service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:23.283661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5155c-5b44-4c1e-a5bd-1b9a15254cce service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:23.284035 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5155c-5b44-4c1e-a5bd-1b9a15254cce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:23.298759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5155c-5b44-4c1e-a5bd-1b9a15254cce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:23.300791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5155c-5b44-4c1e-a5bd-1b9a15254cce service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:23.313796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1107/2153] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:23.349869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b20fa-b7d6-43b1-a60d-5b98c007f341 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:23.450536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b20fa-b7d6-43b1-a60d-5b98c007f341 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:23.464205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:23.465200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:23.465630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:23.465893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] PATH_INFO: `/v3/users/da7299538b9b4e0186db26e7612bfe16` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:23.467877 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=da7299538b9b4e0186db26e7612bfe16)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:23.472148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:23.538177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da7299538b9b4e0186db26e7612bfe16'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:23.548845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da7299538b9b4e0186db26e7612bfe16'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:23.557983 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da7299538b9b4e0186db26e7612bfe16'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:23.577107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da7299538b9b4e0186db26e7612bfe16'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:23.577884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da7299538b9b4e0186db26e7612bfe16'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:23.584542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1047/2154] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:23 2022] DELETE /identity/v3/users/da7299538b9b4e0186db26e7612bfe16 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:23.588886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b86bc-4c75-44dd-9466-1d9554196646 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:23.589180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b86bc-4c75-44dd-9466-1d9554196646 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:23.589501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b86bc-4c75-44dd-9466-1d9554196646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:23.609328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-b54b86bc-4c75-44dd-9466-1d9554196646 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:23.611213 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b54b86bc-4c75-44dd-9466-1d9554196646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:23.619575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:23.620985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:23.665675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:23.672510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1108/2155] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:23.682772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b86bc-4c75-44dd-9466-1d9554196646 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:23.684194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b54b86bc-4c75-44dd-9466-1d9554196646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:23.684325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e95a6-ed1f-45d5-ade8-50a679f0ac0a admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:23.687984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850a038-672c-4362-8e74-324cf1c5e9a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:23.688665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850a038-672c-4362-8e74-324cf1c5e9a7 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:23.688949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850a038-672c-4362-8e74-324cf1c5e9a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:23.689256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850a038-672c-4362-8e74-324cf1c5e9a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:23.691063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d850a038-672c-4362-8e74-324cf1c5e9a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:23.744313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b86bc-4c75-44dd-9466-1d9554196646 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:23.748232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a8799-cde3-49a9-9751-63d996e22891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:23.748878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a8799-cde3-49a9-9751-63d996e22891 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:23.749162 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a8799-cde3-49a9-9751-63d996e22891 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:23.749475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a8799-cde3-49a9-9751-63d996e22891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:23.758589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850a038-672c-4362-8e74-324cf1c5e9a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:23.758957 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a8799-cde3-49a9-9751-63d996e22891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:23.760034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850a038-672c-4362-8e74-324cf1c5e9a7 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:23.760380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a8799-cde3-49a9-9751-63d996e22891 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:23.770640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1109/2157] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:50:23.770810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1048/2157] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:50:24.078323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a8799-cde3-49a9-9751-63d996e22891 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:24.080422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3b9a8799-cde3-49a9-9751-63d996e22891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:24.096195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a8799-cde3-49a9-9751-63d996e22891 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:24.100848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:24.101747 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:24.102033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:24.102326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] PATH_INFO: `/v3/projects/fbb36de9e4fb481da5509fd646eff534` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:24.103584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb36de9e4fb481da5509fd646eff534)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:24.106128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:24.154589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb36de9e4fb481da5509fd646eff534', 'request_id': 'req-1343b10f-f79c-41dc-9381-c301f1f32699'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:24.164623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb36de9e4fb481da5509fd646eff534', 'request_id': 'req-1343b10f-f79c-41dc-9381-c301f1f32699'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:24.171362 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb36de9e4fb481da5509fd646eff534', 'request_id': 'req-1343b10f-f79c-41dc-9381-c301f1f32699'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:24.183056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] The token cache is being invalidate because project fbb36de9e4fb481da5509fd646eff534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:24.183349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:24.184264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1343b10f-f79c-41dc-9381-c301f1f32699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:24.186280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1110/2158] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:24 2022] DELETE /identity/v3/projects/fbb36de9e4fb481da5509fd646eff534 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:27.785841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850a038-672c-4362-8e74-324cf1c5e9a7 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:27.788922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d850a038-672c-4362-8e74-324cf1c5e9a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:27.805032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850a038-672c-4362-8e74-324cf1c5e9a7 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:27.880521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:27.881487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:27.881891 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:27.882174 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] PATH_INFO: `/v3/users/024779d6e6c74835a43d4e68ded23281` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:27.884406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=024779d6e6c74835a43d4e68ded23281)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:27.886853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:27.971521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024779d6e6c74835a43d4e68ded23281'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:27.985733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024779d6e6c74835a43d4e68ded23281'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.006746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024779d6e6c74835a43d4e68ded23281'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.033310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024779d6e6c74835a43d4e68ded23281'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.035466 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024779d6e6c74835a43d4e68ded23281'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.050228 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1049/2159] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:27 2022] DELETE /identity/v3/users/024779d6e6c74835a43d4e68ded23281 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:28.051330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fb370-64fb-468e-b06e-e40140459b69 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:28.051794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fb370-64fb-468e-b06e-e40140459b69 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:28.051893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fb370-64fb-468e-b06e-e40140459b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:28.082621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-687fb370-64fb-468e-b06e-e40140459b69 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:28.084305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-687fb370-64fb-468e-b06e-e40140459b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:28.098101 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:28.188809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1111/2160] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:28.199335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:28.200405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:28.214845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:28.277607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69f030-cd14-4fa2-b57b-11b04181dd87 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:28.281702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1da70f-c2dc-4e81-a329-76b0654a15a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:28.282548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1da70f-c2dc-4e81-a329-76b0654a15a9 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:28.282833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1da70f-c2dc-4e81-a329-76b0654a15a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:28.283109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1da70f-c2dc-4e81-a329-76b0654a15a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:28.294900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1da70f-c2dc-4e81-a329-76b0654a15a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:28.296342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1da70f-c2dc-4e81-a329-76b0654a15a9 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:28.324343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1050/2161] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:28.331241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fb370-64fb-468e-b06e-e40140459b69 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:28.332825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-687fb370-64fb-468e-b06e-e40140459b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:28.345898 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fb370-64fb-468e-b06e-e40140459b69 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:28.351311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:28.352329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:28.352670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:28.353048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] PATH_INFO: `/v3/users/20eed246b8e140688a628664f29e38a4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:28.354403 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20eed246b8e140688a628664f29e38a4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:28.356444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:28.425517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20eed246b8e140688a628664f29e38a4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.441818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20eed246b8e140688a628664f29e38a4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.449602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20eed246b8e140688a628664f29e38a4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.450302 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20eed246b8e140688a628664f29e38a4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.457992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20eed246b8e140688a628664f29e38a4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.464903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1112/2162] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:28 2022] DELETE /identity/v3/users/20eed246b8e140688a628664f29e38a4 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:28.716174 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1da70f-c2dc-4e81-a329-76b0654a15a9 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:28.717791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bd1da70f-c2dc-4e81-a329-76b0654a15a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:28.735758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:28.737876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:28.746633 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1da70f-c2dc-4e81-a329-76b0654a15a9 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:28.810790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:28.811663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:28.812092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:28.812376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] PATH_INFO: `/v3/projects/50cbe058356a4b768c50bcffc605c7e4` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:28.814190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=50cbe058356a4b768c50bcffc605c7e4)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:28.816181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:28.837861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1eb50c-ebe4-439f-a9eb-454e57350447 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:28.844402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:28.845416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:28.845642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:28.846136 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] PATH_INFO: `/v3/projects/0efbb2e43cc74d63ad721e3971e7b181` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:28.847334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0efbb2e43cc74d63ad721e3971e7b181)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:28.849550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:28.879165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50cbe058356a4b768c50bcffc605c7e4', 'request_id': 'req-a875ba22-7af8-4e77-ae9b-3398466061ff'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.886066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50cbe058356a4b768c50bcffc605c7e4', 'request_id': 'req-a875ba22-7af8-4e77-ae9b-3398466061ff'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.889856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:28.893875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50cbe058356a4b768c50bcffc605c7e4', 'request_id': 'req-a875ba22-7af8-4e77-ae9b-3398466061ff'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.907360 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0efbb2e43cc74d63ad721e3971e7b181', 'request_id': 'req-132951a0-bb44-48c3-af70-8b843bc1e89e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.908679 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] The token cache is being invalidate because project 50cbe058356a4b768c50bcffc605c7e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:28.908976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.909671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.911725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1051/2163] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:28 2022] DELETE /identity/v3/projects/50cbe058356a4b768c50bcffc605c7e4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:28.915630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:28.915953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0efbb2e43cc74d63ad721e3971e7b181', 'request_id': 'req-132951a0-bb44-48c3-af70-8b843bc1e89e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.917057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:28.922805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0efbb2e43cc74d63ad721e3971e7b181', 'request_id': 'req-132951a0-bb44-48c3-af70-8b843bc1e89e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.935534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] The token cache is being invalidate because project 0efbb2e43cc74d63ad721e3971e7b181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:28.935830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.936518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-132951a0-bb44-48c3-af70-8b843bc1e89e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:28.939388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1113/2164] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:28 2022] DELETE /identity/v3/projects/0efbb2e43cc74d63ad721e3971e7b181 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:28.947720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ede710-fb58-49d7-a89b-ee6ce4a286a3 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:28.948010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ede710-fb58-49d7-a89b-ee6ce4a286a3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:28.948293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ede710-fb58-49d7-a89b-ee6ce4a286a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:28.976563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-08ede710-fb58-49d7-a89b-ee6ce4a286a3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:28.977664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:28.978861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-08ede710-fb58-49d7-a89b-ee6ce4a286a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:29.011224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875ba22-7af8-4e77-ae9b-3398466061ff admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:29.078819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1114/2165] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:29.080461 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9476c-41ce-4873-978b-c3e66169ea7f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.080860 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9476c-41ce-4873-978b-c3e66169ea7f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.081233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9476c-41ce-4873-978b-c3e66169ea7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.097308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:29.098556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.098848 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.099128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] PATH_INFO: `/v3/users/804d65cd3f4a464ea967ea4ee1f50a18` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.100717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=804d65cd3f4a464ea967ea4ee1f50a18)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:29.105552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-28a9476c-41ce-4873-978b-c3e66169ea7f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:29.105725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:29.105807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-28a9476c-41ce-4873-978b-c3e66169ea7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:29.118011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1115/2166] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:29.122764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9476c-41ce-4873-978b-c3e66169ea7f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:29.133126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9476c-41ce-4873-978b-c3e66169ea7f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:29.138270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ed391-417c-4b81-aa44-1ab353d4fc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:29.139066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ed391-417c-4b81-aa44-1ab353d4fc3b admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.139562 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ed391-417c-4b81-aa44-1ab353d4fc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.139916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ed391-417c-4b81-aa44-1ab353d4fc3b admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.140779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ed391-417c-4b81-aa44-1ab353d4fc3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:29.141304 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ed391-417c-4b81-aa44-1ab353d4fc3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:29.143114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ed391-417c-4b81-aa44-1ab353d4fc3b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:29.157311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1116/2167] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:50:29.165741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ed391-417c-4b81-aa44-1ab353d4fc3b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:29.169961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '804d65cd3f4a464ea967ea4ee1f50a18'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.176784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ed391-417c-4b81-aa44-1ab353d4fc3b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:29.179897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '804d65cd3f4a464ea967ea4ee1f50a18'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.181875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d3f73-fecf-4f70-8e28-ca7b0a12ecdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:29.182533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d3f73-fecf-4f70-8e28-ca7b0a12ecdf admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.182813 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d3f73-fecf-4f70-8e28-ca7b0a12ecdf admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.183088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d3f73-fecf-4f70-8e28-ca7b0a12ecdf admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.183777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d3f73-fecf-4f70-8e28-ca7b0a12ecdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:29.185810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d3f73-fecf-4f70-8e28-ca7b0a12ecdf admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:29.187225 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '804d65cd3f4a464ea967ea4ee1f50a18'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.200029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '804d65cd3f4a464ea967ea4ee1f50a18'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.200751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '804d65cd3f4a464ea967ea4ee1f50a18'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.206939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1052/2168] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:28 2022] DELETE /identity/v3/users/804d65cd3f4a464ea967ea4ee1f50a18 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:29.224917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1117/2169] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:29 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:29.229896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:29.231162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:29.320287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14dc95-00b3-44d0-a6e4-5b7663d017cf admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:29.331915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:29.447227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c5b6d-b6c3-4603-ac1a-b671be0a87fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:29.448356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c5b6d-b6c3-4603-ac1a-b671be0a87fc admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.448938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c5b6d-b6c3-4603-ac1a-b671be0a87fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.449317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c5b6d-b6c3-4603-ac1a-b671be0a87fc admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.450190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c5b6d-b6c3-4603-ac1a-b671be0a87fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:29.453265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c5b6d-b6c3-4603-ac1a-b671be0a87fc admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:29.478884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1053/2170] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:29 2022] POST /identity/v3/users => generated 288 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:50:29.480713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d3f73-fecf-4f70-8e28-ca7b0a12ecdf admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:29.492853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d3f73-fecf-4f70-8e28-ca7b0a12ecdf admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:29.497844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fde66d-4244-4a29-9628-7a4b71a368c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:29.498908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fde66d-4244-4a29-9628-7a4b71a368c3 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.499159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fde66d-4244-4a29-9628-7a4b71a368c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.499940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fde66d-4244-4a29-9628-7a4b71a368c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.501666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fde66d-4244-4a29-9628-7a4b71a368c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:29.503452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fde66d-4244-4a29-9628-7a4b71a368c3 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:29.515861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1118/2171] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:29.520814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c5b6d-b6c3-4603-ac1a-b671be0a87fc admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:29.542315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c5b6d-b6c3-4603-ac1a-b671be0a87fc admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:29.548434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eb8c3-3ce5-456a-a48f-7821a7ad0bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:29.549642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eb8c3-3ce5-456a-a48f-7821a7ad0bfb admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.549642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eb8c3-3ce5-456a-a48f-7821a7ad0bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.549950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eb8c3-3ce5-456a-a48f-7821a7ad0bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.550857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19eb8c3-3ce5-456a-a48f-7821a7ad0bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:29.552717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19eb8c3-3ce5-456a-a48f-7821a7ad0bfb admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:29.569648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1054/2172] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:29.573406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fde66d-4244-4a29-9628-7a4b71a368c3 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:29.588543 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fde66d-4244-4a29-9628-7a4b71a368c3 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:29.592506 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eb8c3-3ce5-456a-a48f-7821a7ad0bfb admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:29.596299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a19eb8c3-3ce5-456a-a48f-7821a7ad0bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:29.600490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ea40e-c2fa-43b3-9818-5dd21b5a8328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:29.602794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ea40e-c2fa-43b3-9818-5dd21b5a8328 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.603107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ea40e-c2fa-43b3-9818-5dd21b5a8328 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.603671 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ea40e-c2fa-43b3-9818-5dd21b5a8328 admin admin] PATH_INFO: `/v3/projects/a35c1526c58f4c2b9b44a4de6c945846/users/cbcb993e2d1842389c5862f0fdb7d4d7/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.619226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eb8c3-3ce5-456a-a48f-7821a7ad0bfb admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:29.624486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:29.625385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.625722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.626175 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] PATH_INFO: `/v3/projects/55ae472d39d341fda42b6e07711a7d74` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.627999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55ae472d39d341fda42b6e07711a7d74)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:29.630164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:29.635057 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ea40e-c2fa-43b3-9818-5dd21b5a8328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a35c1526c58f4c2b9b44a4de6c945846, user_id=cbcb993e2d1842389c5862f0fdb7d4d7, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:29.638778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ea40e-c2fa-43b3-9818-5dd21b5a8328 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:29.662686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1119/2173] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:29 2022] PUT /identity/v3/projects/a35c1526c58f4c2b9b44a4de6c945846/users/cbcb993e2d1842389c5862f0fdb7d4d7/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:29.669019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147389f4-2d65-4013-9570-adf5523edb73 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.669596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147389f4-2d65-4013-9570-adf5523edb73 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.670057 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147389f4-2d65-4013-9570-adf5523edb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.696984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55ae472d39d341fda42b6e07711a7d74', 'request_id': 'req-52e34717-f1bd-4077-86f8-454e86b35a17'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.710244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-147389f4-2d65-4013-9570-adf5523edb73 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:29.710244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-147389f4-2d65-4013-9570-adf5523edb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:29.710454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55ae472d39d341fda42b6e07711a7d74', 'request_id': 'req-52e34717-f1bd-4077-86f8-454e86b35a17'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.722359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55ae472d39d341fda42b6e07711a7d74', 'request_id': 'req-52e34717-f1bd-4077-86f8-454e86b35a17'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.743991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] The token cache is being invalidate because project 55ae472d39d341fda42b6e07711a7d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:29.744545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.745796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.747901 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1055/2174] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:29 2022] DELETE /identity/v3/projects/55ae472d39d341fda42b6e07711a7d74 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:29.754333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:29.758620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:29.780344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:29.818040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1120/2175] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:29.829635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147389f4-2d65-4013-9570-adf5523edb73 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:29.831929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-147389f4-2d65-4013-9570-adf5523edb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:29.873347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e34717-f1bd-4077-86f8-454e86b35a17 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:29.881994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47512c-b4c3-481f-bbaa-2414bb96bf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:29.883856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47512c-b4c3-481f-bbaa-2414bb96bf14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.884340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47512c-b4c3-481f-bbaa-2414bb96bf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.884817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47512c-b4c3-481f-bbaa-2414bb96bf14 admin admin] PATH_INFO: `/v3/users/aabfddc1c5e44ca79ebf119c77375506` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.887246 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47512c-b4c3-481f-bbaa-2414bb96bf14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aabfddc1c5e44ca79ebf119c77375506)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:29.890654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47512c-b4c3-481f-bbaa-2414bb96bf14 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:29.916057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:29.928170 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147389f4-2d65-4013-9570-adf5523edb73 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:29.935107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b99201-7472-499a-806d-30d603b06104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:29.937595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b99201-7472-499a-806d-30d603b06104 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:29.937595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b99201-7472-499a-806d-30d603b06104 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:29.937595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b99201-7472-499a-806d-30d603b06104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:29.951475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b99201-7472-499a-806d-30d603b06104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:29.954597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b99201-7472-499a-806d-30d603b06104 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:29.962216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-5e47512c-b4c3-481f-bbaa-2414bb96bf14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aabfddc1c5e44ca79ebf119c77375506'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.966695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1121/2176] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:50:29.972822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-5e47512c-b4c3-481f-bbaa-2414bb96bf14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aabfddc1c5e44ca79ebf119c77375506'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:29.981616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-5e47512c-b4c3-481f-bbaa-2414bb96bf14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aabfddc1c5e44ca79ebf119c77375506'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:30.005340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-5e47512c-b4c3-481f-bbaa-2414bb96bf14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aabfddc1c5e44ca79ebf119c77375506'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:30.006339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-5e47512c-b4c3-481f-bbaa-2414bb96bf14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aabfddc1c5e44ca79ebf119c77375506'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:30.017379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1056/2177] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:29 2022] DELETE /identity/v3/users/aabfddc1c5e44ca79ebf119c77375506 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:30.348097 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b99201-7472-499a-806d-30d603b06104 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:30.361904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b99201-7472-499a-806d-30d603b06104 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:30.455470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:30.456536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:30.456886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:30.457231 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] PATH_INFO: `/v3/projects/a1f6e185ee414969b562e645000ad48c` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:30.458654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1f6e185ee414969b562e645000ad48c)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:30.460869 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:30.540864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1f6e185ee414969b562e645000ad48c', 'request_id': 'req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:30.556934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1f6e185ee414969b562e645000ad48c', 'request_id': 'req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:30.570634 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1f6e185ee414969b562e645000ad48c', 'request_id': 'req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:30.595989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] The token cache is being invalidate because project a1f6e185ee414969b562e645000ad48c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:30.596447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:30.597661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7ee21f7d-227a-4a9d-850b-0b5c0bc8b8d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:30.601444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1122/2178] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:30 2022] DELETE /identity/v3/projects/a1f6e185ee414969b562e645000ad48c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:30.605467 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd42f4-b3c9-417e-bd12-7629b5e8cc07 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:30.605828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd42f4-b3c9-417e-bd12-7629b5e8cc07 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:30.606172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd42f4-b3c9-417e-bd12-7629b5e8cc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:30.640595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-8ffd42f4-b3c9-417e-bd12-7629b5e8cc07 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:30.642322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8ffd42f4-b3c9-417e-bd12-7629b5e8cc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:30.727711 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1057/2179] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:30.736984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6617b1-f9d3-4153-9c17-526706676920 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:30.737346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6617b1-f9d3-4153-9c17-526706676920 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:30.737581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6617b1-f9d3-4153-9c17-526706676920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:30.764537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-ee6617b1-f9d3-4153-9c17-526706676920 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:30.765870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ee6617b1-f9d3-4153-9c17-526706676920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:30.775026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1123/2180] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:30.783037 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd42f4-b3c9-417e-bd12-7629b5e8cc07 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:30.797057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd42f4-b3c9-417e-bd12-7629b5e8cc07 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:30.814335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324cb58-f487-42b5-bf86-3e99bc24ac23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:30.815743 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324cb58-f487-42b5-bf86-3e99bc24ac23 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:30.816156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324cb58-f487-42b5-bf86-3e99bc24ac23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:30.816442 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324cb58-f487-42b5-bf86-3e99bc24ac23 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:30.817636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324cb58-f487-42b5-bf86-3e99bc24ac23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:30.818091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324cb58-f487-42b5-bf86-3e99bc24ac23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:30.820914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324cb58-f487-42b5-bf86-3e99bc24ac23 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:30.841801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1058/2181] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:50:30.849411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6617b1-f9d3-4153-9c17-526706676920 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:30.871641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6617b1-f9d3-4153-9c17-526706676920 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:30.882579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8afb42-41f0-4333-b3ec-0841d65a0ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:30.883459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8afb42-41f0-4333-b3ec-0841d65a0ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:30.884356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8afb42-41f0-4333-b3ec-0841d65a0ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:30.884356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8afb42-41f0-4333-b3ec-0841d65a0ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:30.884816 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8afb42-41f0-4333-b3ec-0841d65a0ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:30.886996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8afb42-41f0-4333-b3ec-0841d65a0ef4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:30.895767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:30.937420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1124/2182] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:30 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:30.938081 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324cb58-f487-42b5-bf86-3e99bc24ac23 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:30.950627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324cb58-f487-42b5-bf86-3e99bc24ac23 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:31.052946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cc442-6810-460a-9849-7093fd566e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:31.053550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cc442-6810-460a-9849-7093fd566e8e admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:31.053837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cc442-6810-460a-9849-7093fd566e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:31.054154 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cc442-6810-460a-9849-7093fd566e8e admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:31.054817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8cc442-6810-460a-9849-7093fd566e8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:31.056768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8cc442-6810-460a-9849-7093fd566e8e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:31.083506 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1059/2183] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:30 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:31.087706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8afb42-41f0-4333-b3ec-0841d65a0ef4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:31.099971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8afb42-41f0-4333-b3ec-0841d65a0ef4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:31.117827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bed2-fa68-42c2-9ec8-c18964cfab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:31.118533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bed2-fa68-42c2-9ec8-c18964cfab54 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:31.118806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bed2-fa68-42c2-9ec8-c18964cfab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:31.119086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bed2-fa68-42c2-9ec8-c18964cfab54 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:31.119824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364bed2-fa68-42c2-9ec8-c18964cfab54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:31.121456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364bed2-fa68-42c2-9ec8-c18964cfab54 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:31.144257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1125/2184] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:31.145590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cc442-6810-460a-9849-7093fd566e8e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:31.166380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cc442-6810-460a-9849-7093fd566e8e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:31.180118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e12291-e370-42d6-b704-1c3f55aa05e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:31.183585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e12291-e370-42d6-b704-1c3f55aa05e1 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:31.184045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e12291-e370-42d6-b704-1c3f55aa05e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:31.184618 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e12291-e370-42d6-b704-1c3f55aa05e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:31.185796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e12291-e370-42d6-b704-1c3f55aa05e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:31.199025 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e12291-e370-42d6-b704-1c3f55aa05e1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:31.206680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:31.219044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1060/2185] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:50:31.220678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bed2-fa68-42c2-9ec8-c18964cfab54 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:31.233622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bed2-fa68-42c2-9ec8-c18964cfab54 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:31.241834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355e386-f6f0-4298-be07-f292b1478977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:31.245768 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355e386-f6f0-4298-be07-f292b1478977 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:31.246372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355e386-f6f0-4298-be07-f292b1478977 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:31.248504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355e386-f6f0-4298-be07-f292b1478977 admin admin] PATH_INFO: `/v3/projects/0bc1c7a8029e4d40ac256dd2f877317b/users/6f3e4ee4b3ca492cabf2622f0659b5a5/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:31.271258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355e386-f6f0-4298-be07-f292b1478977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bc1c7a8029e4d40ac256dd2f877317b, user_id=6f3e4ee4b3ca492cabf2622f0659b5a5, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:31.274377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355e386-f6f0-4298-be07-f292b1478977 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:31.299141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1126/2186] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:31 2022] PUT /identity/v3/projects/0bc1c7a8029e4d40ac256dd2f877317b/users/6f3e4ee4b3ca492cabf2622f0659b5a5/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:31.301708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff1ad8-102d-42e5-8fe9-c715896cacec None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:31.302026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff1ad8-102d-42e5-8fe9-c715896cacec None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:31.302345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff1ad8-102d-42e5-8fe9-c715896cacec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:31.321943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-bdff1ad8-102d-42e5-8fe9-c715896cacec None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:31.325339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bdff1ad8-102d-42e5-8fe9-c715896cacec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:31.443109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1061/2187] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:31.456274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355e386-f6f0-4298-be07-f292b1478977 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:31.458964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c355e386-f6f0-4298-be07-f292b1478977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:31.546366 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355e386-f6f0-4298-be07-f292b1478977 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:31.554830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa3877-3c06-45c3-89b1-b52615ce2434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:31.556003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3877-3c06-45c3-89b1-b52615ce2434 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:31.556540 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3877-3c06-45c3-89b1-b52615ce2434 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:31.556977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3877-3c06-45c3-89b1-b52615ce2434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:31.568994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa3877-3c06-45c3-89b1-b52615ce2434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:31.570744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa3877-3c06-45c3-89b1-b52615ce2434 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:31.588500 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1127/2188] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:33.222181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff1ad8-102d-42e5-8fe9-c715896cacec None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:33.224798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bdff1ad8-102d-42e5-8fe9-c715896cacec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:33.286801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff1ad8-102d-42e5-8fe9-c715896cacec None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:33.305179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:33.306742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:33.307051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:33.307338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] PATH_INFO: `/v3/users/fde425631fc14e7e9d9bbcadd5e31dc7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:33.309863 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde425631fc14e7e9d9bbcadd5e31dc7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:33.312425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:33.451172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde425631fc14e7e9d9bbcadd5e31dc7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:33.476189 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde425631fc14e7e9d9bbcadd5e31dc7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:33.487232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde425631fc14e7e9d9bbcadd5e31dc7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:33.509841 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde425631fc14e7e9d9bbcadd5e31dc7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:33.510645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde425631fc14e7e9d9bbcadd5e31dc7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:33.526976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1062/2189] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:33 2022] DELETE /identity/v3/users/fde425631fc14e7e9d9bbcadd5e31dc7 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:33.627477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdd908-f73e-4a61-af11-44a31587408b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:33.629287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdd908-f73e-4a61-af11-44a31587408b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:33.629826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bdd908-f73e-4a61-af11-44a31587408b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:33.668780 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-78bdd908-f73e-4a61-af11-44a31587408b None None] MFA Rules not processed for user `6f3e4ee4b3ca492cabf2622f0659b5a5`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:33.670834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-78bdd908-f73e-4a61-af11-44a31587408b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:33.754988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1128/2190] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:33.766987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:33.768885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:33.899705 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bdd908-f73e-4a61-af11-44a31587408b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:33.901954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-78bdd908-f73e-4a61-af11-44a31587408b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:33.904388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634ddbd-dce2-4838-a8ce-d962a91e1e1a admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:33.909300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cc11a-24f6-4ece-b907-7e16cbdb7756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:33.910242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cc11a-24f6-4ece-b907-7e16cbdb7756 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:33.910692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cc11a-24f6-4ece-b907-7e16cbdb7756 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:33.911417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cc11a-24f6-4ece-b907-7e16cbdb7756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:33.932502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cc11a-24f6-4ece-b907-7e16cbdb7756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:33.935045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cc11a-24f6-4ece-b907-7e16cbdb7756 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:33.950655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1063/2191] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:50:34.038484 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:34.075268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bdd908-f73e-4a61-af11-44a31587408b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:34.081620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:34.082313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:34.082596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:34.082879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] PATH_INFO: `/v3/projects/29a23eb116194101b7577fa0692864d9` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:34.084176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a23eb116194101b7577fa0692864d9)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:34.086708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:34.187932 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a23eb116194101b7577fa0692864d9', 'request_id': 'req-257d5404-c0fb-4608-8e5c-4457677fda4b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:34.202636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a23eb116194101b7577fa0692864d9', 'request_id': 'req-257d5404-c0fb-4608-8e5c-4457677fda4b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:34.219182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a23eb116194101b7577fa0692864d9', 'request_id': 'req-257d5404-c0fb-4608-8e5c-4457677fda4b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:34.254507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] The token cache is being invalidate because project 29a23eb116194101b7577fa0692864d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:34.255072 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:34.257458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:34.263766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1129/2192] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:33 2022] DELETE /identity/v3/projects/29a23eb116194101b7577fa0692864d9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:34.266073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cc11a-24f6-4ece-b907-7e16cbdb7756 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:34.294916 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cc11a-24f6-4ece-b907-7e16cbdb7756 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:34.327739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:34.455689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6283f6d-3956-4d6b-a9bf-0a3848059657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:34.456679 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283f6d-3956-4d6b-a9bf-0a3848059657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:34.457044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283f6d-3956-4d6b-a9bf-0a3848059657 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:34.458103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6283f6d-3956-4d6b-a9bf-0a3848059657 admin admin] PATH_INFO: `/v3/users/162e45c1d1f74988ba322d4be5f6b97b` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:34.459472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6283f6d-3956-4d6b-a9bf-0a3848059657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=162e45c1d1f74988ba322d4be5f6b97b)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:34.462431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6283f6d-3956-4d6b-a9bf-0a3848059657 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:34.573685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a6283f6d-3956-4d6b-a9bf-0a3848059657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162e45c1d1f74988ba322d4be5f6b97b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:34.591971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a6283f6d-3956-4d6b-a9bf-0a3848059657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162e45c1d1f74988ba322d4be5f6b97b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:34.602662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a6283f6d-3956-4d6b-a9bf-0a3848059657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162e45c1d1f74988ba322d4be5f6b97b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:34.629032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a6283f6d-3956-4d6b-a9bf-0a3848059657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162e45c1d1f74988ba322d4be5f6b97b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:34.630020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a6283f6d-3956-4d6b-a9bf-0a3848059657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162e45c1d1f74988ba322d4be5f6b97b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:34.646837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1064/2193] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:34 2022] DELETE /identity/v3/users/162e45c1d1f74988ba322d4be5f6b97b => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:34.990601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:34.992917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:35.017641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d5404-c0fb-4608-8e5c-4457677fda4b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:35.119373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:35.120549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:35.120935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:35.121282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] PATH_INFO: `/v3/projects/cc46f8938bd3497cb765277a941475f2` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:35.122958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc46f8938bd3497cb765277a941475f2)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:35.125247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:35.237903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc46f8938bd3497cb765277a941475f2', 'request_id': 'req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:35.250989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc46f8938bd3497cb765277a941475f2', 'request_id': 'req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:35.261725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc46f8938bd3497cb765277a941475f2', 'request_id': 'req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:35.287507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] The token cache is being invalidate because project cc46f8938bd3497cb765277a941475f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:35.289617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:35.290930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-67be2032-8cbe-4e7a-aacd-5616fe1ec94a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:35.294596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1130/2194] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:34 2022] DELETE /identity/v3/projects/cc46f8938bd3497cb765277a941475f2 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:35.300707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa532fcf-af29-4f56-93e7-204ae5dc6049 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:35.301264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa532fcf-af29-4f56-93e7-204ae5dc6049 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:35.301538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa532fcf-af29-4f56-93e7-204ae5dc6049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:35.341161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-aa532fcf-af29-4f56-93e7-204ae5dc6049 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:35.343103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-aa532fcf-af29-4f56-93e7-204ae5dc6049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:35.459388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1065/2195] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:35.469715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97434803-7e19-4cdf-91d1-639eac87294f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:35.470326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97434803-7e19-4cdf-91d1-639eac87294f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:35.471663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97434803-7e19-4cdf-91d1-639eac87294f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:35.484017 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:35.500880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-97434803-7e19-4cdf-91d1-639eac87294f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:35.503586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-97434803-7e19-4cdf-91d1-639eac87294f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:35.517098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1131/2196] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:35.522380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa532fcf-af29-4f56-93e7-204ae5dc6049 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:35.538236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa532fcf-af29-4f56-93e7-204ae5dc6049 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:35.552417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19150db-6fb3-4d91-a280-c9ceab2d2f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:35.553165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19150db-6fb3-4d91-a280-c9ceab2d2f9e admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:35.553483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19150db-6fb3-4d91-a280-c9ceab2d2f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:35.553764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19150db-6fb3-4d91-a280-c9ceab2d2f9e admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:35.560832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19150db-6fb3-4d91-a280-c9ceab2d2f9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:35.561159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19150db-6fb3-4d91-a280-c9ceab2d2f9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:35.562896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19150db-6fb3-4d91-a280-c9ceab2d2f9e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:35.591949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1066/2197] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:50:35.593625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97434803-7e19-4cdf-91d1-639eac87294f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:35.609797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97434803-7e19-4cdf-91d1-639eac87294f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:35.615621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08994c1-17c8-4e26-b095-e0395bfccd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:35.616508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08994c1-17c8-4e26-b095-e0395bfccd37 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:35.616899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08994c1-17c8-4e26-b095-e0395bfccd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:35.617393 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08994c1-17c8-4e26-b095-e0395bfccd37 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:35.618646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08994c1-17c8-4e26-b095-e0395bfccd37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:35.620322 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08994c1-17c8-4e26-b095-e0395bfccd37 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:35.667680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1132/2198] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:35 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:35.671808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19150db-6fb3-4d91-a280-c9ceab2d2f9e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:35.684332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19150db-6fb3-4d91-a280-c9ceab2d2f9e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:35.707577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:35.758482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24747-cc22-4fc6-80d6-8c4681d737c0 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:35.758772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24747-cc22-4fc6-80d6-8c4681d737c0 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:35.759094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24747-cc22-4fc6-80d6-8c4681d737c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:35.785238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cdab6-6f11-4d09-a6ea-bd222b6bc4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:35.786053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cdab6-6f11-4d09-a6ea-bd222b6bc4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:35.786490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cdab6-6f11-4d09-a6ea-bd222b6bc4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:35.786772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cdab6-6f11-4d09-a6ea-bd222b6bc4b5 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:35.787496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cdab6-6f11-4d09-a6ea-bd222b6bc4b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:35.789788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cdab6-6f11-4d09-a6ea-bd222b6bc4b5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:35.795923 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-d9b24747-cc22-4fc6-80d6-8c4681d737c0 None None] MFA Rules not processed for user `6f3e4ee4b3ca492cabf2622f0659b5a5`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:35.797192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d9b24747-cc22-4fc6-80d6-8c4681d737c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:35.829659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1067/2199] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:35 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:35.830836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cdab6-6f11-4d09-a6ea-bd222b6bc4b5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:35.849812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cdab6-6f11-4d09-a6ea-bd222b6bc4b5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:35.858661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490596d-e3e3-4c16-b98f-28402d6b5c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:35.859355 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490596d-e3e3-4c16-b98f-28402d6b5c41 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:35.859644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490596d-e3e3-4c16-b98f-28402d6b5c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:35.860512 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490596d-e3e3-4c16-b98f-28402d6b5c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:35.861259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490596d-e3e3-4c16-b98f-28402d6b5c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:35.862934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490596d-e3e3-4c16-b98f-28402d6b5c41 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:35.900988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1068/2200] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:35.902699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490596d-e3e3-4c16-b98f-28402d6b5c41 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:35.926137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490596d-e3e3-4c16-b98f-28402d6b5c41 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:35.940219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901782ce-8e1e-4a9c-83ea-5366fbbb4891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:35.940919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901782ce-8e1e-4a9c-83ea-5366fbbb4891 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:35.941204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901782ce-8e1e-4a9c-83ea-5366fbbb4891 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:35.941517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901782ce-8e1e-4a9c-83ea-5366fbbb4891 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:35.942262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901782ce-8e1e-4a9c-83ea-5366fbbb4891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:35.943869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901782ce-8e1e-4a9c-83ea-5366fbbb4891 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:35.966738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1133/2201] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:35.972205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1069/2202] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:35.973784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b24747-cc22-4fc6-80d6-8c4681d737c0 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:36.008494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b24747-cc22-4fc6-80d6-8c4681d737c0 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:36.028311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20f94b-f658-4621-a6c3-180d335f7599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:36.029580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20f94b-f658-4621-a6c3-180d335f7599 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:36.029987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20f94b-f658-4621-a6c3-180d335f7599 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:36.030269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20f94b-f658-4621-a6c3-180d335f7599 admin admin] PATH_INFO: `/v3/projects/2e0ff8171d7b4987a4c646c51bcf878d/users/adee497377fc48fd9c97f7b8867fbd31/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:36.072282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20f94b-f658-4621-a6c3-180d335f7599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0ff8171d7b4987a4c646c51bcf878d, user_id=adee497377fc48fd9c97f7b8867fbd31, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:36.076921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20f94b-f658-4621-a6c3-180d335f7599 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:36.152757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1134/2203] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:35 2022] PUT /identity/v3/projects/2e0ff8171d7b4987a4c646c51bcf878d/users/adee497377fc48fd9c97f7b8867fbd31/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:36.157296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cc2d4-27df-44e0-850c-b945252ffc03 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:36.157643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cc2d4-27df-44e0-850c-b945252ffc03 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:36.157893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cc2d4-27df-44e0-850c-b945252ffc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:36.229308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a41cc2d4-27df-44e0-850c-b945252ffc03 None None] MFA Rules not processed for user `adee497377fc48fd9c97f7b8867fbd31`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:36.231540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a41cc2d4-27df-44e0-850c-b945252ffc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:36.393754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1070/2204] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:36.405632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20f94b-f658-4621-a6c3-180d335f7599 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:36.408487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2b20f94b-f658-4621-a6c3-180d335f7599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:36.496613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20f94b-f658-4621-a6c3-180d335f7599 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:36.503507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da13b3c-d1d3-4329-ac5d-d639fb893314 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82e1192b2d554199a08debb4aa392479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:36.504454 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13b3c-d1d3-4329-ac5d-d639fb893314 service glance] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:36.504831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13b3c-d1d3-4329-ac5d-d639fb893314 service glance] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:36.505309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13b3c-d1d3-4329-ac5d-d639fb893314 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:36.526564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da13b3c-d1d3-4329-ac5d-d639fb893314 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:36.528716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da13b3c-d1d3-4329-ac5d-d639fb893314 service glance] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:36.572676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1135/2205] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:37.090306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce178a-0ab0-4751-8c59-14fc85efb8ae None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:37.090881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce178a-0ab0-4751-8c59-14fc85efb8ae None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:37.091549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce178a-0ab0-4751-8c59-14fc85efb8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:37.174899 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a3ce178a-0ab0-4751-8c59-14fc85efb8ae None None] MFA Rules not processed for user `cbcb993e2d1842389c5862f0fdb7d4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:37.180295 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a3ce178a-0ab0-4751-8c59-14fc85efb8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:37.397148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1071/2206] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:37.412052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da13b3c-d1d3-4329-ac5d-d639fb893314 service glance] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:37.414660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1da13b3c-d1d3-4329-ac5d-d639fb893314 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:37.614138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:37.652279 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da13b3c-d1d3-4329-ac5d-d639fb893314 service glance] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:37.668524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f826f-b0c2-4e02-889b-d8a97c6caa79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:37.679625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f826f-b0c2-4e02-889b-d8a97c6caa79 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:37.682061 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f826f-b0c2-4e02-889b-d8a97c6caa79 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:37.682428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f826f-b0c2-4e02-889b-d8a97c6caa79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:37.698800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f826f-b0c2-4e02-889b-d8a97c6caa79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:37.700645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f826f-b0c2-4e02-889b-d8a97c6caa79 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:37.715447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1136/2207] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:39.521003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e6336-f56b-4512-910c-e9f55c84aa32 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:39.522786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e6336-f56b-4512-910c-e9f55c84aa32 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:39.523483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e6336-f56b-4512-910c-e9f55c84aa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:39.543236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-464e6336-f56b-4512-910c-e9f55c84aa32 None None] MFA Rules not processed for user `cbcb993e2d1842389c5862f0fdb7d4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:39.544659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-464e6336-f56b-4512-910c-e9f55c84aa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:39.557451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1072/2208] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:47.709282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=7 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:48.177738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f826f-b0c2-4e02-889b-d8a97c6caa79 service nova] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:48.179188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d17f826f-b0c2-4e02-889b-d8a97c6caa79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:48.245307 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f826f-b0c2-4e02-889b-d8a97c6caa79 service nova] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:48.249868 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:48.250735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:48.251016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:48.251291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] PATH_INFO: `/v3/users/72b980470adc4ba0b391f70219eca8ab` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:48.252464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b980470adc4ba0b391f70219eca8ab)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:48.254491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:48.302742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b980470adc4ba0b391f70219eca8ab'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:48.315501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b980470adc4ba0b391f70219eca8ab'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:48.323394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b980470adc4ba0b391f70219eca8ab'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:48.324099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b980470adc4ba0b391f70219eca8ab'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:48.334022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b980470adc4ba0b391f70219eca8ab'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:48.341093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1137/2209] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:48 2022] DELETE /identity/v3/users/72b980470adc4ba0b391f70219eca8ab => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:48.346081 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae47b7-5195-4f9c-a4fb-33b598e81bd1 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:48.346547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae47b7-5195-4f9c-a4fb-33b598e81bd1 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:48.346867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae47b7-5195-4f9c-a4fb-33b598e81bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:48.354383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:48.381441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-32ae47b7-5195-4f9c-a4fb-33b598e81bd1 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:48.382861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-32ae47b7-5195-4f9c-a4fb-33b598e81bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:48.455226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1073/2210] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:48.470839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:48.472354 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:48.533749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564237f-f630-4c88-bfaa-b952dde69651 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:48.545096 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7f8f4-e638-4420-b139-512f6be67fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:48.546008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7f8f4-e638-4420-b139-512f6be67fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:48.546343 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7f8f4-e638-4420-b139-512f6be67fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:48.546625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7f8f4-e638-4420-b139-512f6be67fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:48.558350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7f8f4-e638-4420-b139-512f6be67fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:48.559798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7f8f4-e638-4420-b139-512f6be67fb9 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:48.569852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1138/2211] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:48.919205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae47b7-5195-4f9c-a4fb-33b598e81bd1 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:48.920777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-32ae47b7-5195-4f9c-a4fb-33b598e81bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:48.934171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae47b7-5195-4f9c-a4fb-33b598e81bd1 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:48.940193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:48.941088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:48.941430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:48.941800 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] PATH_INFO: `/v3/projects/fa251f62e3ba456ba3d8a23b9f137afb` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:48.943778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa251f62e3ba456ba3d8a23b9f137afb)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:48.946123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:49.028190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa251f62e3ba456ba3d8a23b9f137afb', 'request_id': 'req-cc283be5-33f1-4b4a-a046-bdbac5825e66'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:49.034897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa251f62e3ba456ba3d8a23b9f137afb', 'request_id': 'req-cc283be5-33f1-4b4a-a046-bdbac5825e66'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:49.042124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa251f62e3ba456ba3d8a23b9f137afb', 'request_id': 'req-cc283be5-33f1-4b4a-a046-bdbac5825e66'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:49.058923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] The token cache is being invalidate because project fa251f62e3ba456ba3d8a23b9f137afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:49.059214 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:49.059922 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc283be5-33f1-4b4a-a046-bdbac5825e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:49.061685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1074/2212] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:48 2022] DELETE /identity/v3/projects/fa251f62e3ba456ba3d8a23b9f137afb => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:49.067305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e64a7-6811-44a4-8bab-c12a7ce7c474 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:49.068530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e64a7-6811-44a4-8bab-c12a7ce7c474 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:49.068530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e64a7-6811-44a4-8bab-c12a7ce7c474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:49.090740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-263e64a7-6811-44a4-8bab-c12a7ce7c474 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:49.091982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-263e64a7-6811-44a4-8bab-c12a7ce7c474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:49.115008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:49.164421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1139/2213] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:49.172941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92c593-a3a3-49c3-9af7-dceb06724de6 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:49.173238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92c593-a3a3-49c3-9af7-dceb06724de6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:49.173560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92c593-a3a3-49c3-9af7-dceb06724de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:49.193970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-6f92c593-a3a3-49c3-9af7-dceb06724de6 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:49.195188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6f92c593-a3a3-49c3-9af7-dceb06724de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:49.203254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1075/2214] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:49.208391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e64a7-6811-44a4-8bab-c12a7ce7c474 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:49.222918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e64a7-6811-44a4-8bab-c12a7ce7c474 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:49.228143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46866c86-046c-44ec-ad1d-37a5239c0263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:49.228880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46866c86-046c-44ec-ad1d-37a5239c0263 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:49.229260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46866c86-046c-44ec-ad1d-37a5239c0263 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:49.229639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46866c86-046c-44ec-ad1d-37a5239c0263 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:49.230441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46866c86-046c-44ec-ad1d-37a5239c0263 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:49.230825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46866c86-046c-44ec-ad1d-37a5239c0263 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:49.232481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46866c86-046c-44ec-ad1d-37a5239c0263 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:49.245911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1140/2215] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:50:49.250106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92c593-a3a3-49c3-9af7-dceb06724de6 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:49.260118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92c593-a3a3-49c3-9af7-dceb06724de6 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:49.264785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238affe-f94d-4887-9a52-21647b59b9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:49.265470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238affe-f94d-4887-9a52-21647b59b9eb admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:49.265758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238affe-f94d-4887-9a52-21647b59b9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:49.266144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238affe-f94d-4887-9a52-21647b59b9eb admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:49.266955 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238affe-f94d-4887-9a52-21647b59b9eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:49.269461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238affe-f94d-4887-9a52-21647b59b9eb admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:49.303699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1076/2216] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:49 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:49.306020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46866c86-046c-44ec-ad1d-37a5239c0263 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:49.317127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46866c86-046c-44ec-ad1d-37a5239c0263 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:49.379898 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd43f26-653f-441a-a06b-429c91136aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:49.380593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd43f26-653f-441a-a06b-429c91136aad admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:49.380879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd43f26-653f-441a-a06b-429c91136aad admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:49.381165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd43f26-653f-441a-a06b-429c91136aad admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:49.381925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd43f26-653f-441a-a06b-429c91136aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:49.383884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd43f26-653f-441a-a06b-429c91136aad admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:49.406321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1141/2217] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:49 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:50:49.410685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238affe-f94d-4887-9a52-21647b59b9eb admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:49.423990 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238affe-f94d-4887-9a52-21647b59b9eb admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:49.429563 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53994095-bfc9-486d-9cae-256475d0b9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:49.430245 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53994095-bfc9-486d-9cae-256475d0b9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:49.430532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53994095-bfc9-486d-9cae-256475d0b9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:49.430804 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53994095-bfc9-486d-9cae-256475d0b9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:49.431535 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53994095-bfc9-486d-9cae-256475d0b9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:49.433269 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53994095-bfc9-486d-9cae-256475d0b9c1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:49.445283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1077/2218] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:49.449722 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd43f26-653f-441a-a06b-429c91136aad admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:49.460761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd43f26-653f-441a-a06b-429c91136aad admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:49.465872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f28c2-c6cb-4aa8-89be-7371bd6c24d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:49.466552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f28c2-c6cb-4aa8-89be-7371bd6c24d9 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:49.466842 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f28c2-c6cb-4aa8-89be-7371bd6c24d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:49.467131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f28c2-c6cb-4aa8-89be-7371bd6c24d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:49.467875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f28c2-c6cb-4aa8-89be-7371bd6c24d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:49.469817 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f28c2-c6cb-4aa8-89be-7371bd6c24d9 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:49.482052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1142/2219] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:49.487066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53994095-bfc9-486d-9cae-256475d0b9c1 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:49.497751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53994095-bfc9-486d-9cae-256475d0b9c1 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:49.503325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84885ba2-6227-4059-b0a3-b697912e26a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:49.504357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84885ba2-6227-4059-b0a3-b697912e26a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:49.504714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84885ba2-6227-4059-b0a3-b697912e26a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:49.505066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84885ba2-6227-4059-b0a3-b697912e26a4 admin admin] PATH_INFO: `/v3/projects/5285b463d71240629eb90b5251f4373f/users/bfc46df6f92c4bc1a81c9f88ca366057/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:49.521515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84885ba2-6227-4059-b0a3-b697912e26a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5285b463d71240629eb90b5251f4373f, user_id=bfc46df6f92c4bc1a81c9f88ca366057, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:49.524207 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84885ba2-6227-4059-b0a3-b697912e26a4 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:49.538909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1078/2220] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:49 2022] PUT /identity/v3/projects/5285b463d71240629eb90b5251f4373f/users/bfc46df6f92c4bc1a81c9f88ca366057/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:50:49.550620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ac1c0-8b82-4bba-8662-a52a76e00f94 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:49.551311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ac1c0-8b82-4bba-8662-a52a76e00f94 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:49.551881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ac1c0-8b82-4bba-8662-a52a76e00f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:49.587493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-3f2ac1c0-8b82-4bba-8662-a52a76e00f94 None None] MFA Rules not processed for user `bfc46df6f92c4bc1a81c9f88ca366057`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:49.588916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3f2ac1c0-8b82-4bba-8662-a52a76e00f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:49.662373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1143/2221] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:50:49.674367 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84885ba2-6227-4059-b0a3-b697912e26a4 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:49.676104 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-84885ba2-6227-4059-b0a3-b697912e26a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:49.703369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:49.738836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84885ba2-6227-4059-b0a3-b697912e26a4 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:49.743072 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd6ba9-0a0c-46c2-9b9d-d2e50d87c8e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:49.743901 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd6ba9-0a0c-46c2-9b9d-d2e50d87c8e1 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:49.744194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd6ba9-0a0c-46c2-9b9d-d2e50d87c8e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:49.744476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd6ba9-0a0c-46c2-9b9d-d2e50d87c8e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:49.753807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd6ba9-0a0c-46c2-9b9d-d2e50d87c8e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:49.755244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd6ba9-0a0c-46c2-9b9d-d2e50d87c8e1 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:49.764264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1079/2222] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:51.156931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f4aa-c673-4e21-ba0d-ebf94420438f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:51.157544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f4aa-c673-4e21-ba0d-ebf94420438f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:51.157544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f4aa-c673-4e21-ba0d-ebf94420438f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:51.175740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-fbc4f4aa-c673-4e21-ba0d-ebf94420438f None None] MFA Rules not processed for user `bfc46df6f92c4bc1a81c9f88ca366057`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:51.176976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fbc4f4aa-c673-4e21-ba0d-ebf94420438f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:51.184735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1144/2223] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:50:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:59.760172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=16 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:01.170312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:05.392886 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509e96c-423a-4926-a489-379810837101 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:05.393503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509e96c-423a-4926-a489-379810837101 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:05.393503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509e96c-423a-4926-a489-379810837101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:05.412213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-4509e96c-423a-4926-a489-379810837101 None None] MFA Rules not processed for user `6f3e4ee4b3ca492cabf2622f0659b5a5`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:05.413505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4509e96c-423a-4926-a489-379810837101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:05.475765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1080/2224] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:51:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:07.784396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4f4aa-c673-4e21-ba0d-ebf94420438f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:07.785887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fbc4f4aa-c673-4e21-ba0d-ebf94420438f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:07.864390 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4f4aa-c673-4e21-ba0d-ebf94420438f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:07.869662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:07.870572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:07.870852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:07.871128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] PATH_INFO: `/v3/users/adee497377fc48fd9c97f7b8867fbd31` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:07.872361 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adee497377fc48fd9c97f7b8867fbd31)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:07.874363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:07.937345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adee497377fc48fd9c97f7b8867fbd31'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:07.954984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adee497377fc48fd9c97f7b8867fbd31'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:07.966140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adee497377fc48fd9c97f7b8867fbd31'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:07.966894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adee497377fc48fd9c97f7b8867fbd31'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:07.977011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adee497377fc48fd9c97f7b8867fbd31'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:07.985272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1145/2225] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:51:07 2022] DELETE /identity/v3/users/adee497377fc48fd9c97f7b8867fbd31 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:51:07.990169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa6224-8a7d-45a8-992e-b488ad520ab5 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:07.992097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa6224-8a7d-45a8-992e-b488ad520ab5 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:07.992893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa6224-8a7d-45a8-992e-b488ad520ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:08.015106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-31fa6224-8a7d-45a8-992e-b488ad520ab5 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:08.016412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-31fa6224-8a7d-45a8-992e-b488ad520ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:08.090026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1081/2226] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:08.110214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:08.112586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:08.180150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214464b-d62c-4291-ad07-fa142bc31822 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:08.183985 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58928797-764c-40ef-ab34-83a0579b238c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:08.184642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58928797-764c-40ef-ab34-83a0579b238c service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:08.184928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58928797-764c-40ef-ab34-83a0579b238c service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:08.185206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58928797-764c-40ef-ab34-83a0579b238c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:08.196437 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58928797-764c-40ef-ab34-83a0579b238c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:08.197932 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58928797-764c-40ef-ab34-83a0579b238c service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:08.208801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1146/2227] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:51:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:51:08.489741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa6224-8a7d-45a8-992e-b488ad520ab5 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:08.491229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-31fa6224-8a7d-45a8-992e-b488ad520ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:08.504320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa6224-8a7d-45a8-992e-b488ad520ab5 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:08.509816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:08.510708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:08.510995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:08.511277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] PATH_INFO: `/v3/projects/2e0ff8171d7b4987a4c646c51bcf878d` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:08.512537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0ff8171d7b4987a4c646c51bcf878d)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:08.514520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:08.564463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0ff8171d7b4987a4c646c51bcf878d', 'request_id': 'req-c03c8f00-d521-4749-86e4-b55fb5448387'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:08.571016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0ff8171d7b4987a4c646c51bcf878d', 'request_id': 'req-c03c8f00-d521-4749-86e4-b55fb5448387'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:08.576636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0ff8171d7b4987a4c646c51bcf878d', 'request_id': 'req-c03c8f00-d521-4749-86e4-b55fb5448387'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:08.590801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] The token cache is being invalidate because project 2e0ff8171d7b4987a4c646c51bcf878d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:51:08.591099 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:08.591748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c03c8f00-d521-4749-86e4-b55fb5448387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:08.593345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1082/2228] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:51:08 2022] DELETE /identity/v3/projects/2e0ff8171d7b4987a4c646c51bcf878d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:51:08.598904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2f2ab-7270-4dfb-88d8-de4a4542db8b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:08.599272 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2f2ab-7270-4dfb-88d8-de4a4542db8b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:08.599637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2f2ab-7270-4dfb-88d8-de4a4542db8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:08.619688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-3bd2f2ab-7270-4dfb-88d8-de4a4542db8b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:08.620824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3bd2f2ab-7270-4dfb-88d8-de4a4542db8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:08.638476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:08.684860 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1147/2229] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:08.693435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8561-b41a-461d-9e0b-8370e1136491 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:08.693733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8561-b41a-461d-9e0b-8370e1136491 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:08.694031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8561-b41a-461d-9e0b-8370e1136491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:08.713496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-4bcf8561-b41a-461d-9e0b-8370e1136491 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:08.714636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4bcf8561-b41a-461d-9e0b-8370e1136491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:08.722927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1083/2230] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:08.727419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2f2ab-7270-4dfb-88d8-de4a4542db8b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:08.737978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2f2ab-7270-4dfb-88d8-de4a4542db8b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:08.742653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc51ab1-8bb3-46b6-9cc0-af33e74846f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:08.743310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc51ab1-8bb3-46b6-9cc0-af33e74846f0 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:08.743596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc51ab1-8bb3-46b6-9cc0-af33e74846f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:08.743877 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc51ab1-8bb3-46b6-9cc0-af33e74846f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:08.744612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc51ab1-8bb3-46b6-9cc0-af33e74846f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:51:08.744924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc51ab1-8bb3-46b6-9cc0-af33e74846f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:08.746564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc51ab1-8bb3-46b6-9cc0-af33e74846f0 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:08.759011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1148/2231] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:51:08.763545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf8561-b41a-461d-9e0b-8370e1136491 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:08.773444 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf8561-b41a-461d-9e0b-8370e1136491 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:08.778664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29692eb-7a45-473b-b2dd-a23ec47ec94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:08.779328 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29692eb-7a45-473b-b2dd-a23ec47ec94c admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:08.779653 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29692eb-7a45-473b-b2dd-a23ec47ec94c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:08.779890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29692eb-7a45-473b-b2dd-a23ec47ec94c admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:08.780595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29692eb-7a45-473b-b2dd-a23ec47ec94c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:08.782525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29692eb-7a45-473b-b2dd-a23ec47ec94c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:08.811386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:08.829606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1084/2232] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:51:08 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:51:08.831411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc51ab1-8bb3-46b6-9cc0-af33e74846f0 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:08.848871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc51ab1-8bb3-46b6-9cc0-af33e74846f0 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:08.918730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54787d1a-d509-4cc4-88e8-24ae16f0f9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:08.919501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54787d1a-d509-4cc4-88e8-24ae16f0f9ae admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:08.919854 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54787d1a-d509-4cc4-88e8-24ae16f0f9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:08.920357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54787d1a-d509-4cc4-88e8-24ae16f0f9ae admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:08.921155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54787d1a-d509-4cc4-88e8-24ae16f0f9ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:08.924642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54787d1a-d509-4cc4-88e8-24ae16f0f9ae admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:08.952208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1149/2233] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:51:08 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:51:08.956986 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29692eb-7a45-473b-b2dd-a23ec47ec94c admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:08.977539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29692eb-7a45-473b-b2dd-a23ec47ec94c admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:08.982755 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf796059-cfc0-4b55-a5f5-468dd8c67c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:08.983429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf796059-cfc0-4b55-a5f5-468dd8c67c1b admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:08.983720 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf796059-cfc0-4b55-a5f5-468dd8c67c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:08.984002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf796059-cfc0-4b55-a5f5-468dd8c67c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:08.985282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf796059-cfc0-4b55-a5f5-468dd8c67c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:08.986940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf796059-cfc0-4b55-a5f5-468dd8c67c1b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:08.999378 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1085/2234] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:51:09.006025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54787d1a-d509-4cc4-88e8-24ae16f0f9ae admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:09.017664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54787d1a-d509-4cc4-88e8-24ae16f0f9ae admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:09.022821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04bc79-0f95-4816-b2e2-793d07cc2c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:09.023492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04bc79-0f95-4816-b2e2-793d07cc2c24 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:09.023793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04bc79-0f95-4816-b2e2-793d07cc2c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:09.024070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04bc79-0f95-4816-b2e2-793d07cc2c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:09.024809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04bc79-0f95-4816-b2e2-793d07cc2c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:09.026434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04bc79-0f95-4816-b2e2-793d07cc2c24 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:09.042206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1150/2235] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:51:09.043311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf796059-cfc0-4b55-a5f5-468dd8c67c1b admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:09.054380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf796059-cfc0-4b55-a5f5-468dd8c67c1b admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:09.059312 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d8084-eb63-4cc7-964c-81d4b02ee728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:09.060201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8084-eb63-4cc7-964c-81d4b02ee728 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:09.060489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8084-eb63-4cc7-964c-81d4b02ee728 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:09.060793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8084-eb63-4cc7-964c-81d4b02ee728 admin admin] PATH_INFO: `/v3/projects/3ce9a6fb4835451db5acbf33fe715c97/users/f76719a935fe4cdbbf78ef4c746c6dc1/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:09.076293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d8084-eb63-4cc7-964c-81d4b02ee728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce9a6fb4835451db5acbf33fe715c97, user_id=f76719a935fe4cdbbf78ef4c746c6dc1, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:09.079015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d8084-eb63-4cc7-964c-81d4b02ee728 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:09.092775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1086/2236] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:51:09 2022] PUT /identity/v3/projects/3ce9a6fb4835451db5acbf33fe715c97/users/f76719a935fe4cdbbf78ef4c746c6dc1/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:51:09.106523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dece46-de75-45a5-8fd1-6b43becedd5c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:09.106870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dece46-de75-45a5-8fd1-6b43becedd5c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:09.107155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dece46-de75-45a5-8fd1-6b43becedd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:09.136648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-55dece46-de75-45a5-8fd1-6b43becedd5c None None] MFA Rules not processed for user `f76719a935fe4cdbbf78ef4c746c6dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:09.137859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-55dece46-de75-45a5-8fd1-6b43becedd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:09.215109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1151/2237] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:51:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:51:09.228343 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d8084-eb63-4cc7-964c-81d4b02ee728 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:09.229737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-198d8084-eb63-4cc7-964c-81d4b02ee728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:09.311744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d8084-eb63-4cc7-964c-81d4b02ee728 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:09.316090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b18f1-5b83-4447-9a40-901cb08963dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:09.316780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b18f1-5b83-4447-9a40-901cb08963dc service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:09.317065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b18f1-5b83-4447-9a40-901cb08963dc service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:09.317345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b18f1-5b83-4447-9a40-901cb08963dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:09.326741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b18f1-5b83-4447-9a40-901cb08963dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:09.328520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b18f1-5b83-4447-9a40-901cb08963dc service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:09.338859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1087/2238] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:51:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:51:18.383937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdbb90-7122-4ddf-bbad-65bb9dff2148 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:18.384477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdbb90-7122-4ddf-bbad-65bb9dff2148 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:18.384477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fdbb90-7122-4ddf-bbad-65bb9dff2148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:18.402536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e8fdbb90-7122-4ddf-bbad-65bb9dff2148 None None] MFA Rules not processed for user `9fa96f51390e46e991c6e9afa385d2f7`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:18.403738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e8fdbb90-7122-4ddf-bbad-65bb9dff2148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:18.417099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:18.464029 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1152/2239] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:51:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:19.335399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:19.723280 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b18f1-5b83-4447-9a40-901cb08963dc service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:19.795495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b18f1-5b83-4447-9a40-901cb08963dc service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:19.800932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:19.801838 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:19.802128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:19.802409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] PATH_INFO: `/v3/users/f76719a935fe4cdbbf78ef4c746c6dc1` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:19.803630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76719a935fe4cdbbf78ef4c746c6dc1)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:19.806069 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:19.868382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76719a935fe4cdbbf78ef4c746c6dc1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:19.880849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76719a935fe4cdbbf78ef4c746c6dc1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:19.895682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76719a935fe4cdbbf78ef4c746c6dc1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:19.913829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76719a935fe4cdbbf78ef4c746c6dc1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:19.914989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76719a935fe4cdbbf78ef4c746c6dc1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:19.927206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1088/2240] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:51:19 2022] DELETE /identity/v3/users/f76719a935fe4cdbbf78ef4c746c6dc1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:51:19.933972 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71560f3a-5fd4-4973-9c4a-e6671a050594 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:19.934502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71560f3a-5fd4-4973-9c4a-e6671a050594 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:19.934871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71560f3a-5fd4-4973-9c4a-e6671a050594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:19.967159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-71560f3a-5fd4-4973-9c4a-e6671a050594 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:19.969285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-71560f3a-5fd4-4973-9c4a-e6671a050594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:20.059256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1153/2241] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:20.070253 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:20.071698 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:20.138554 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08c84a-885d-411e-917a-06f4e1b3f478 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:20.142717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf59ccc-7ae9-49c8-b3d7-4a73fbc28032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:20.143394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf59ccc-7ae9-49c8-b3d7-4a73fbc28032 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:20.143724 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf59ccc-7ae9-49c8-b3d7-4a73fbc28032 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:20.143960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf59ccc-7ae9-49c8-b3d7-4a73fbc28032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:20.156553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf59ccc-7ae9-49c8-b3d7-4a73fbc28032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:20.158043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf59ccc-7ae9-49c8-b3d7-4a73fbc28032 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:20.171272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1089/2242] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:51:20.546899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71560f3a-5fd4-4973-9c4a-e6671a050594 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:20.551180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-71560f3a-5fd4-4973-9c4a-e6671a050594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:20.598776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71560f3a-5fd4-4973-9c4a-e6671a050594 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:20.598776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:20.598776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:20.598776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:20.600566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] PATH_INFO: `/v3/projects/3ce9a6fb4835451db5acbf33fe715c97` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:20.600566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce9a6fb4835451db5acbf33fe715c97)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:20.602580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:20.705953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce9a6fb4835451db5acbf33fe715c97', 'request_id': 'req-ea598f83-a615-4fcd-a907-6958b2f6d8cf'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:20.722497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce9a6fb4835451db5acbf33fe715c97', 'request_id': 'req-ea598f83-a615-4fcd-a907-6958b2f6d8cf'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:20.754004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce9a6fb4835451db5acbf33fe715c97', 'request_id': 'req-ea598f83-a615-4fcd-a907-6958b2f6d8cf'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:20.769024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] The token cache is being invalidate because project 3ce9a6fb4835451db5acbf33fe715c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:51:20.769345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:20.770748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:20.772703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1154/2243] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:51:20 2022] DELETE /identity/v3/projects/3ce9a6fb4835451db5acbf33fe715c97 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:51:30.165143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:30.664581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f3515-7e82-4f1a-b8d1-6f8121f9eec9 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:30.666461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f3515-7e82-4f1a-b8d1-6f8121f9eec9 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:30.667176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f3515-7e82-4f1a-b8d1-6f8121f9eec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:30.697042 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-da8f3515-7e82-4f1a-b8d1-6f8121f9eec9 None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:30.699090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-da8f3515-7e82-4f1a-b8d1-6f8121f9eec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:30.764692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:30.778874 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1090/2244] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:51:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:30.838924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:30.842093 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:30.964025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea598f83-a615-4fcd-a907-6958b2f6d8cf admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:30.970641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8671c-196d-4bf1-ab58-3a71ae0544de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:30.971325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8671c-196d-4bf1-ab58-3a71ae0544de service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:30.971621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8671c-196d-4bf1-ab58-3a71ae0544de service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:30.971925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8671c-196d-4bf1-ab58-3a71ae0544de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:30.982375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8671c-196d-4bf1-ab58-3a71ae0544de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:30.984034 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8671c-196d-4bf1-ab58-3a71ae0544de service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:30.995631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1155/2245] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:51:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:51:31.812591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba459abd-ba18-4ad7-8255-0152496c852d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:31.813126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba459abd-ba18-4ad7-8255-0152496c852d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:31.813279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba459abd-ba18-4ad7-8255-0152496c852d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:31.833759 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ba459abd-ba18-4ad7-8255-0152496c852d None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:31.834923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ba459abd-ba18-4ad7-8255-0152496c852d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:31.842775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1091/2246] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jul 19 20:51:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:31.863578 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8671c-196d-4bf1-ab58-3a71ae0544de service swift] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:31.875332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8671c-196d-4bf1-ab58-3a71ae0544de service swift] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:31.882159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb778b7-63e3-4847-b978-3de520bc3225 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:31.882874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb778b7-63e3-4847-b978-3de520bc3225 service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:31.883198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb778b7-63e3-4847-b978-3de520bc3225 service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:31.883529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb778b7-63e3-4847-b978-3de520bc3225 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:31.894899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb778b7-63e3-4847-b978-3de520bc3225 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:31.896658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb778b7-63e3-4847-b978-3de520bc3225 service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:31.907460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1156/2247] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:51:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:51:32.168232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd50c7-d72e-4030-a340-f865640c616d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:32.168525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd50c7-d72e-4030-a340-f865640c616d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:32.168804 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd50c7-d72e-4030-a340-f865640c616d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:32.188538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-06cd50c7-d72e-4030-a340-f865640c616d None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:32.189813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-06cd50c7-d72e-4030-a340-f865640c616d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:32.197948 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1092/2248] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:51:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:32.211385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb778b7-63e3-4847-b978-3de520bc3225 service swift] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:32.222812 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb778b7-63e3-4847-b978-3de520bc3225 service swift] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:32.228299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7526f6e-18a4-4b8b-9117-8d19d32259ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:32.228963 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7526f6e-18a4-4b8b-9117-8d19d32259ed service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:32.229257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7526f6e-18a4-4b8b-9117-8d19d32259ed service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:32.229580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7526f6e-18a4-4b8b-9117-8d19d32259ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:32.239426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7526f6e-18a4-4b8b-9117-8d19d32259ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:32.241033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7526f6e-18a4-4b8b-9117-8d19d32259ed service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:32.251262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1157/2249] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:51:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 20:51:32.744615 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866267f-fc9d-464d-9b64-caf3c234de5e None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:32.744959 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866267f-fc9d-464d-9b64-caf3c234de5e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:32.745405 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866267f-fc9d-464d-9b64-caf3c234de5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:32.771020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-2866267f-fc9d-464d-9b64-caf3c234de5e None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:32.772327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2866267f-fc9d-464d-9b64-caf3c234de5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:32.785475 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1093/2250] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jul 19 20:51:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:51:32.805833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7526f6e-18a4-4b8b-9117-8d19d32259ed service swift] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:32.829526 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7526f6e-18a4-4b8b-9117-8d19d32259ed service swift] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:32.841593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d228cb9-2f3c-4292-a6c1-ca5e34b92384 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:32.842485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d228cb9-2f3c-4292-a6c1-ca5e34b92384 service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:32.842964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d228cb9-2f3c-4292-a6c1-ca5e34b92384 service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:32.843709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d228cb9-2f3c-4292-a6c1-ca5e34b92384 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:32.862494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d228cb9-2f3c-4292-a6c1-ca5e34b92384 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:32.866149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d228cb9-2f3c-4292-a6c1-ca5e34b92384 service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:32.884157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1158/2251] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:51:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:51:39.762912 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba881eb-8004-439f-88d8-770c84c9a64c None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:39.763583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba881eb-8004-439f-88d8-770c84c9a64c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:39.763583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba881eb-8004-439f-88d8-770c84c9a64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:39.785100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-8ba881eb-8004-439f-88d8-770c84c9a64c None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:39.786896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8ba881eb-8004-439f-88d8-770c84c9a64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:39.795081 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1094/2252] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:51:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:51:39.809240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d228cb9-2f3c-4292-a6c1-ca5e34b92384 service swift] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:39.830174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d228cb9-2f3c-4292-a6c1-ca5e34b92384 service swift] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:39.839357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6c530-f563-4559-9802-476cdb3457d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:39.840448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6c530-f563-4559-9802-476cdb3457d8 service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:39.840961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6c530-f563-4559-9802-476cdb3457d8 service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:39.841676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6c530-f563-4559-9802-476cdb3457d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:39.857839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6c530-f563-4559-9802-476cdb3457d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:39.861622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6c530-f563-4559-9802-476cdb3457d8 service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:39.877141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1159/2253] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:51:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:51:40.023032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba881eb-8004-439f-88d8-770c84c9a64c None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:40.025195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8ba881eb-8004-439f-88d8-770c84c9a64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:40.162159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba881eb-8004-439f-88d8-770c84c9a64c None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:40.174499 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:40.177025 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:40.177508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:40.178856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] PATH_INFO: `/v3/users/bfc46df6f92c4bc1a81c9f88ca366057` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:40.182625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc46df6f92c4bc1a81c9f88ca366057)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:40.190002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:40.311556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc46df6f92c4bc1a81c9f88ca366057'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:40.331684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc46df6f92c4bc1a81c9f88ca366057'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:40.366543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc46df6f92c4bc1a81c9f88ca366057'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:40.393956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc46df6f92c4bc1a81c9f88ca366057'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:40.395167 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc46df6f92c4bc1a81c9f88ca366057'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:40.397742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:40.409057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1095/2254] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:51:40 2022] DELETE /identity/v3/users/bfc46df6f92c4bc1a81c9f88ca366057 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:51:40.416154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30bc46-bbb2-494a-8799-bc18e3704cab None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:40.416495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30bc46-bbb2-494a-8799-bc18e3704cab None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:40.416719 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30bc46-bbb2-494a-8799-bc18e3704cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:40.440323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-8b30bc46-bbb2-494a-8799-bc18e3704cab None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:40.442009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8b30bc46-bbb2-494a-8799-bc18e3704cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:40.539030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1160/2255] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:51:40.559707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:40.561944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:40.763439 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961a9d6-bc5e-4083-847f-908653985582 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:40.775592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423545e-ec46-482f-b1f7-d536063f9f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:40.776790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423545e-ec46-482f-b1f7-d536063f9f6e service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:40.777289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423545e-ec46-482f-b1f7-d536063f9f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:40.777896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423545e-ec46-482f-b1f7-d536063f9f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:40.803439 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423545e-ec46-482f-b1f7-d536063f9f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:40.806166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423545e-ec46-482f-b1f7-d536063f9f6e service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:40.825238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1096/2256] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:51:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:51:41.210141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30bc46-bbb2-494a-8799-bc18e3704cab None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:41.212868 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8b30bc46-bbb2-494a-8799-bc18e3704cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:41.240048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30bc46-bbb2-494a-8799-bc18e3704cab None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:41.247241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:41.248376 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:41.266278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:41.266861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] PATH_INFO: `/v3/projects/5285b463d71240629eb90b5251f4373f` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:41.268888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5285b463d71240629eb90b5251f4373f)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:41.272495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:41.354819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:41.388022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5285b463d71240629eb90b5251f4373f', 'request_id': 'req-2dcef770-c173-4964-88ed-22beb4d7168d'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:41.400051 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5285b463d71240629eb90b5251f4373f', 'request_id': 'req-2dcef770-c173-4964-88ed-22beb4d7168d'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:41.408328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5285b463d71240629eb90b5251f4373f', 'request_id': 'req-2dcef770-c173-4964-88ed-22beb4d7168d'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:41.425572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] The token cache is being invalidate because project 5285b463d71240629eb90b5251f4373f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:51:41.425572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:41.426676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-2dcef770-c173-4964-88ed-22beb4d7168d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:51:41.432766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1161/2257] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:51:41 2022] DELETE /identity/v3/projects/5285b463d71240629eb90b5251f4373f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:51:41.440866 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904acd50-8a87-4e39-8ed3-68a13af6707f None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:41.441309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904acd50-8a87-4e39-8ed3-68a13af6707f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:41.442744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904acd50-8a87-4e39-8ed3-68a13af6707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:41.475684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-904acd50-8a87-4e39-8ed3-68a13af6707f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:41.478130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-904acd50-8a87-4e39-8ed3-68a13af6707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:41.623036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1097/2258] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:41.633204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ab65-de11-4ed3-9d86-f4da91e38c31 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:41.633680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ab65-de11-4ed3-9d86-f4da91e38c31 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:41.634324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ab65-de11-4ed3-9d86-f4da91e38c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:41.669295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-5de8ab65-de11-4ed3-9d86-f4da91e38c31 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:41.671527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5de8ab65-de11-4ed3-9d86-f4da91e38c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:41.686083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1162/2259] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:41.692217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904acd50-8a87-4e39-8ed3-68a13af6707f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:41.712531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904acd50-8a87-4e39-8ed3-68a13af6707f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:41.721788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e587c-8d89-4570-bbc7-e19910d85e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:41.723011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e587c-8d89-4570-bbc7-e19910d85e23 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:41.723710 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e587c-8d89-4570-bbc7-e19910d85e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:41.724125 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e587c-8d89-4570-bbc7-e19910d85e23 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:41.725567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e587c-8d89-4570-bbc7-e19910d85e23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:51:41.726511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e587c-8d89-4570-bbc7-e19910d85e23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:41.729525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e587c-8d89-4570-bbc7-e19910d85e23 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:41.755275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1098/2260] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:51:41.762125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8ab65-de11-4ed3-9d86-f4da91e38c31 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:41.780135 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8ab65-de11-4ed3-9d86-f4da91e38c31 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:41.788204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d77f7-300e-4d67-9981-fe74c94cd573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:41.789420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d77f7-300e-4d67-9981-fe74c94cd573 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:41.790010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d77f7-300e-4d67-9981-fe74c94cd573 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:41.790498 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d77f7-300e-4d67-9981-fe74c94cd573 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:41.791798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d77f7-300e-4d67-9981-fe74c94cd573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:41.797250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d77f7-300e-4d67-9981-fe74c94cd573 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:41.852885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1163/2261] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:51:41 2022] POST /identity/v3/projects => generated 391 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:51:41.858106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e587c-8d89-4570-bbc7-e19910d85e23 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:41.881344 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e587c-8d89-4570-bbc7-e19910d85e23 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:42.007721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5915b1-8312-4a59-913e-1484a720afb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:42.009443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5915b1-8312-4a59-913e-1484a720afb1 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:42.010263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5915b1-8312-4a59-913e-1484a720afb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:42.010773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5915b1-8312-4a59-913e-1484a720afb1 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:42.011982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5915b1-8312-4a59-913e-1484a720afb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:42.016193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5915b1-8312-4a59-913e-1484a720afb1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:42.051708 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1099/2262] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:51:41 2022] POST /identity/v3/users => generated 296 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:51:42.058015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d77f7-300e-4d67-9981-fe74c94cd573 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:42.081679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d77f7-300e-4d67-9981-fe74c94cd573 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:42.091119 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d45f3a-fda9-462d-945b-b48b2cc6e740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:42.092194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d45f3a-fda9-462d-945b-b48b2cc6e740 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:42.092545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d45f3a-fda9-462d-945b-b48b2cc6e740 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:42.093027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d45f3a-fda9-462d-945b-b48b2cc6e740 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:42.095360 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d45f3a-fda9-462d-945b-b48b2cc6e740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:42.098395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d45f3a-fda9-462d-945b-b48b2cc6e740 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:42.130872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1164/2263] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:51:42.132956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5915b1-8312-4a59-913e-1484a720afb1 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:42.155023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5915b1-8312-4a59-913e-1484a720afb1 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:42.164138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab982ff9-3e5f-4632-a92a-78489c47e5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:42.165203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab982ff9-3e5f-4632-a92a-78489c47e5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:42.165669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab982ff9-3e5f-4632-a92a-78489c47e5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:42.166150 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab982ff9-3e5f-4632-a92a-78489c47e5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:42.167538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab982ff9-3e5f-4632-a92a-78489c47e5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:42.169345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab982ff9-3e5f-4632-a92a-78489c47e5d1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:42.187126 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1100/2264] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:51:42.191635 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d45f3a-fda9-462d-945b-b48b2cc6e740 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:42.208041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d45f3a-fda9-462d-945b-b48b2cc6e740 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:42.213739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ecd44-a354-41e2-bf2a-fbdbace90984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:42.214906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ecd44-a354-41e2-bf2a-fbdbace90984 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:42.214906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ecd44-a354-41e2-bf2a-fbdbace90984 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:42.215212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ecd44-a354-41e2-bf2a-fbdbace90984 admin admin] PATH_INFO: `/v3/projects/5dffc4cca1a8451c99dd574c3bada98e/users/01fb9594cbf043d7a8a4fa9e7ef2af6f/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:42.241754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ecd44-a354-41e2-bf2a-fbdbace90984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dffc4cca1a8451c99dd574c3bada98e, user_id=01fb9594cbf043d7a8a4fa9e7ef2af6f, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:42.247036 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ecd44-a354-41e2-bf2a-fbdbace90984 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:42.289043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1165/2265] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:51:42 2022] PUT /identity/v3/projects/5dffc4cca1a8451c99dd574c3bada98e/users/01fb9594cbf043d7a8a4fa9e7ef2af6f/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:51:42.301731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcffdaa-3456-48a5-a979-7edde026077c None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:42.302613 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcffdaa-3456-48a5-a979-7edde026077c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:42.303473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcffdaa-3456-48a5-a979-7edde026077c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:42.328721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:42.362679 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-fdcffdaa-3456-48a5-a979-7edde026077c None None] MFA Rules not processed for user `01fb9594cbf043d7a8a4fa9e7ef2af6f`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:51:42.364972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-fdcffdaa-3456-48a5-a979-7edde026077c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:42.558054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1101/2266] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:51:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:51:42.595877 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ecd44-a354-41e2-bf2a-fbdbace90984 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:51:42.600716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-777ecd44-a354-41e2-bf2a-fbdbace90984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:51:42.714462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ecd44-a354-41e2-bf2a-fbdbace90984 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:51:42.720276 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15a795-db51-4f9a-b506-c0b0f3961c01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:51:42.721548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15a795-db51-4f9a-b506-c0b0f3961c01 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:51:42.721947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15a795-db51-4f9a-b506-c0b0f3961c01 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:51:42.722553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15a795-db51-4f9a-b506-c0b0f3961c01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:51:42.742664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15a795-db51-4f9a-b506-c0b0f3961c01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:51:42.745876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15a795-db51-4f9a-b506-c0b0f3961c01 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:51:42.769641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1166/2267] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:51:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:51:52.549092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=29 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:51:52.761314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:52:03.513210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bcb92-167a-4013-8259-d7df9213e7f9 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:03.514064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bcb92-167a-4013-8259-d7df9213e7f9 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:03.514064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bcb92-167a-4013-8259-d7df9213e7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:03.533723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-bd7bcb92-167a-4013-8259-d7df9213e7f9 None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:03.538212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bd7bcb92-167a-4013-8259-d7df9213e7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:03.636718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1102/2268] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jul 19 20:52:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:52:03.655551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15a795-db51-4f9a-b506-c0b0f3961c01 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:03.658065 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8a15a795-db51-4f9a-b506-c0b0f3961c01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:03.792462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15a795-db51-4f9a-b506-c0b0f3961c01 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:03.799025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd4189-20cb-4199-a36b-39de86020f65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:03.799767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd4189-20cb-4199-a36b-39de86020f65 service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:03.800050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd4189-20cb-4199-a36b-39de86020f65 service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:03.800324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd4189-20cb-4199-a36b-39de86020f65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:03.812931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd4189-20cb-4199-a36b-39de86020f65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:03.814848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd4189-20cb-4199-a36b-39de86020f65 service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:03.825748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1167/2269] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:52:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:52:06.304524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bcb92-167a-4013-8259-d7df9213e7f9 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:06.307260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bd7bcb92-167a-4013-8259-d7df9213e7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:06.431508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bcb92-167a-4013-8259-d7df9213e7f9 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:06.437438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:06.438699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:06.439191 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:06.439676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] PATH_INFO: `/v3/users/6f3e4ee4b3ca492cabf2622f0659b5a5` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:06.442197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f3e4ee4b3ca492cabf2622f0659b5a5)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:06.444590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:06.516509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f3e4ee4b3ca492cabf2622f0659b5a5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:06.531529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f3e4ee4b3ca492cabf2622f0659b5a5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:06.546117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f3e4ee4b3ca492cabf2622f0659b5a5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:06.569651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f3e4ee4b3ca492cabf2622f0659b5a5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:06.570829 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f3e4ee4b3ca492cabf2622f0659b5a5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:06.582845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1103/2270] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:52:06 2022] DELETE /identity/v3/users/6f3e4ee4b3ca492cabf2622f0659b5a5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:52:06.883671 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd4189-20cb-4199-a36b-39de86020f65 service swift] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:06.904966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd4189-20cb-4199-a36b-39de86020f65 service swift] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:07.022546 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:07.024084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:07.024559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:07.025038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] PATH_INFO: `/v3/projects/0bc1c7a8029e4d40ac256dd2f877317b` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:07.027690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bc1c7a8029e4d40ac256dd2f877317b)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:07.031758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:07.136897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bc1c7a8029e4d40ac256dd2f877317b', 'request_id': 'req-8980674d-5ec8-4ad9-be3e-a60ba7036c93'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:07.151236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bc1c7a8029e4d40ac256dd2f877317b', 'request_id': 'req-8980674d-5ec8-4ad9-be3e-a60ba7036c93'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:07.168695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bc1c7a8029e4d40ac256dd2f877317b', 'request_id': 'req-8980674d-5ec8-4ad9-be3e-a60ba7036c93'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:07.192663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] The token cache is being invalidate because project 0bc1c7a8029e4d40ac256dd2f877317b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:52:07.193184 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:07.194422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8980674d-5ec8-4ad9-be3e-a60ba7036c93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:07.198011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1168/2271] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:52:06 2022] DELETE /identity/v3/projects/0bc1c7a8029e4d40ac256dd2f877317b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:52:09.117178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:09.119775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:09.144813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac197d-4207-4ad9-aac4-8f8a89f6fb52 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:09.183585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:52:09.237274 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:09.238308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:09.238642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:09.238970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] PATH_INFO: `/v3/users/01fb9594cbf043d7a8a4fa9e7ef2af6f` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:09.240436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01fb9594cbf043d7a8a4fa9e7ef2af6f)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:09.242602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:09.309793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01fb9594cbf043d7a8a4fa9e7ef2af6f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:09.320897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01fb9594cbf043d7a8a4fa9e7ef2af6f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:09.331317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01fb9594cbf043d7a8a4fa9e7ef2af6f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:09.350605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01fb9594cbf043d7a8a4fa9e7ef2af6f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:09.351781 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01fb9594cbf043d7a8a4fa9e7ef2af6f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:09.363521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1104/2272] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:52:09 2022] DELETE /identity/v3/users/01fb9594cbf043d7a8a4fa9e7ef2af6f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:52:09.370583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4df2-b326-4814-a51a-916256689a92 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:09.371447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4df2-b326-4814-a51a-916256689a92 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:09.371603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4df2-b326-4814-a51a-916256689a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:09.404254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e6bc4df2-b326-4814-a51a-916256689a92 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:09.406133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e6bc4df2-b326-4814-a51a-916256689a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:09.425743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:52:09.501048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1169/2273] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:09.514436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:09.517084 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:09.612058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd00e1-0b3e-4915-81b0-9ef23515ed63 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:09.619205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c948ed-34c9-433d-97f3-8db7ba0d4034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:09.620342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c948ed-34c9-433d-97f3-8db7ba0d4034 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:09.620865 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c948ed-34c9-433d-97f3-8db7ba0d4034 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:09.621415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c948ed-34c9-433d-97f3-8db7ba0d4034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:09.639665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c948ed-34c9-433d-97f3-8db7ba0d4034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:09.642734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c948ed-34c9-433d-97f3-8db7ba0d4034 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:09.662067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1105/2274] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:52:10.063076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc4df2-b326-4814-a51a-916256689a92 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:10.066037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e6bc4df2-b326-4814-a51a-916256689a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:10.085275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc4df2-b326-4814-a51a-916256689a92 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:10.090595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:10.091488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:10.091769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:10.092042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] PATH_INFO: `/v3/projects/5dffc4cca1a8451c99dd574c3bada98e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:10.094033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dffc4cca1a8451c99dd574c3bada98e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:10.096049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:10.170644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dffc4cca1a8451c99dd574c3bada98e', 'request_id': 'req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:10.181239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dffc4cca1a8451c99dd574c3bada98e', 'request_id': 'req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:10.191259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dffc4cca1a8451c99dd574c3bada98e', 'request_id': 'req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:10.212073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] The token cache is being invalidate because project 5dffc4cca1a8451c99dd574c3bada98e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:52:10.212350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:10.213189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:52:10.214911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1170/2275] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:52:10 2022] DELETE /identity/v3/projects/5dffc4cca1a8451c99dd574c3bada98e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:52:19.653251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=68,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:52:20.207895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:52:41.141307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b137-9613-40ba-8c35-65186b2b46fd None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:41.142334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b137-9613-40ba-8c35-65186b2b46fd None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:41.142511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264b137-9613-40ba-8c35-65186b2b46fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:41.171347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0264b137-9613-40ba-8c35-65186b2b46fd None None] MFA Rules not processed for user `af989ff6dc024ff19023e6f633915ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:41.172612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0264b137-9613-40ba-8c35-65186b2b46fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:41.241556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1106/2276] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:52:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:42.814401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:42.816861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:42.944527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15f9c0-55a8-4eeb-ad9a-be8d9fd832c2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:42.950415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cff0b2-d751-4ec5-8aeb-2556ace3b9fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:42.951147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cff0b2-d751-4ec5-8aeb-2556ace3b9fb service placement] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:42.951478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cff0b2-d751-4ec5-8aeb-2556ace3b9fb service placement] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:42.951798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cff0b2-d751-4ec5-8aeb-2556ace3b9fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:42.954951 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-48cff0b2-d751-4ec5-8aeb-2556ace3b9fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:42.971417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cff0b2-d751-4ec5-8aeb-2556ace3b9fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:42.973706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cff0b2-d751-4ec5-8aeb-2556ace3b9fb service placement] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:42.987138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1171/2277] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:52:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:52:51.233657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=25 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:52:52.980800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=32 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:53:01.750394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a685-c10a-48f3-ba16-f66de28e37c0 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:01.751003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a685-c10a-48f3-ba16-f66de28e37c0 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:01.751107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a685-c10a-48f3-ba16-f66de28e37c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:01.784808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-1897a685-c10a-48f3-ba16-f66de28e37c0 None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:53:01.786645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1897a685-c10a-48f3-ba16-f66de28e37c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:01.862180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1107/2278] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:53:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:53:01.890436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cff0b2-d751-4ec5-8aeb-2556ace3b9fb service placement] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:53:01.891969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-48cff0b2-d751-4ec5-8aeb-2556ace3b9fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:01.995317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cff0b2-d751-4ec5-8aeb-2556ace3b9fb service placement] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:53:02.002702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d245a8-db25-4233-9051-579e0e4e81ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:53:02.003776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d245a8-db25-4233-9051-579e0e4e81ab service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:02.004244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d245a8-db25-4233-9051-579e0e4e81ab service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:02.004725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d245a8-db25-4233-9051-579e0e4e81ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:02.017262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d245a8-db25-4233-9051-579e0e4e81ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:53:02.020140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d245a8-db25-4233-9051-579e0e4e81ab service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:53:02.035767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1172/2279] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:53:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:53:11.854256 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=25 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:53:12.032847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=32 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:53:17.502503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484806d-43d4-4c57-b014-4888a7ce1375 None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:17.503459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484806d-43d4-4c57-b014-4888a7ce1375 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:17.503459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484806d-43d4-4c57-b014-4888a7ce1375 None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:17.506458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1108/2280] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 20:53:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:53:17.520110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e538d02-c763-4fb9-a883-fcea0b170240 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:17.520739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e538d02-c763-4fb9-a883-fcea0b170240 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:17.520739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e538d02-c763-4fb9-a883-fcea0b170240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:17.545083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-6e538d02-c763-4fb9-a883-fcea0b170240 None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:53:17.547176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6e538d02-c763-4fb9-a883-fcea0b170240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:17.666397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1173/2281] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 20:53:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:53:17.685903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484806d-43d4-4c57-b014-4888a7ce1375 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:53:17.689825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d484806d-43d4-4c57-b014-4888a7ce1375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:17.816215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484806d-43d4-4c57-b014-4888a7ce1375 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:53:17.822674 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1a18e-62a0-4596-922c-2b9a36fd61d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82e1192b2d554199a08debb4aa392479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:53:17.823874 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1a18e-62a0-4596-922c-2b9a36fd61d6 service glance] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:17.824378 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1a18e-62a0-4596-922c-2b9a36fd61d6 service glance] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:17.824858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1a18e-62a0-4596-922c-2b9a36fd61d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:17.847764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1a18e-62a0-4596-922c-2b9a36fd61d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:53:17.849630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1a18e-62a0-4596-922c-2b9a36fd61d6 service glance] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:53:17.863991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1109/2282] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:53:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:53:17.953432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a040a29-dc48-46a7-8de8-f571eb76a61e None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:17.953731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a040a29-dc48-46a7-8de8-f571eb76a61e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:17.954219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a040a29-dc48-46a7-8de8-f571eb76a61e None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:17.955834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1174/2283] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 20:53:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:53:17.962258 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225365d4-3b5a-416e-9dce-f6aa7d7ca3ec None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:17.962523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225365d4-3b5a-416e-9dce-f6aa7d7ca3ec None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:17.962993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225365d4-3b5a-416e-9dce-f6aa7d7ca3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:17.995476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-225365d4-3b5a-416e-9dce-f6aa7d7ca3ec None None] MFA Rules not processed for user `fb186ee2e3c94ca7ad203c9c0e32e498`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:53:17.997206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-225365d4-3b5a-416e-9dce-f6aa7d7ca3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:18.070330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1110/2284] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 20:53:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:53:18.087099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a040a29-dc48-46a7-8de8-f571eb76a61e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:53:18.089020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0a040a29-dc48-46a7-8de8-f571eb76a61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:18.103730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a040a29-dc48-46a7-8de8-f571eb76a61e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:53:18.107454 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251257ba-1823-4183-b76c-3888f66c4571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:53:18.108733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251257ba-1823-4183-b76c-3888f66c4571 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:18.109020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251257ba-1823-4183-b76c-3888f66c4571 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:18.109309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251257ba-1823-4183-b76c-3888f66c4571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:18.121697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251257ba-1823-4183-b76c-3888f66c4571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:53:18.123312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251257ba-1823-4183-b76c-3888f66c4571 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:53:18.133112 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1175/2285] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:53:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:53:18.256840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c53af-8fac-48d6-953b-faa26f42715c None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:18.257309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c53af-8fac-48d6-953b-faa26f42715c None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:18.257722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c53af-8fac-48d6-953b-faa26f42715c None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:18.259425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1111/2286] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 20:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:53:18.268912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbf7d3-57c8-43c0-8a5e-bcd533bd783a None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:18.269244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbf7d3-57c8-43c0-8a5e-bcd533bd783a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:18.269615 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbf7d3-57c8-43c0-8a5e-bcd533bd783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:18.290303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e4cbf7d3-57c8-43c0-8a5e-bcd533bd783a None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:53:18.291570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e4cbf7d3-57c8-43c0-8a5e-bcd533bd783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:18.302226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1176/2287] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 20:53:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:53:18.314618 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c53af-8fac-48d6-953b-faa26f42715c None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:53:18.327092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c53af-8fac-48d6-953b-faa26f42715c None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:53:18.331396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323b75d-eb78-466a-8477-d9db05ef8dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:53:18.332106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323b75d-eb78-466a-8477-d9db05ef8dda service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:18.332407 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323b75d-eb78-466a-8477-d9db05ef8dda service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:18.332714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323b75d-eb78-466a-8477-d9db05ef8dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:18.344965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323b75d-eb78-466a-8477-d9db05ef8dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:53:18.346456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323b75d-eb78-466a-8477-d9db05ef8dda service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:53:18.360609 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1112/2288] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:53:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:53:21.522386 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0db15-a3f3-4870-a3a0-78235d017527 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:21.523561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0db15-a3f3-4870-a3a0-78235d017527 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:21.524069 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0db15-a3f3-4870-a3a0-78235d017527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:21.546558 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-c4e0db15-a3f3-4870-a3a0-78235d017527 None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:53:21.547992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c4e0db15-a3f3-4870-a3a0-78235d017527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:21.561507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1177/2289] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:53:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:53:21.571699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323b75d-eb78-466a-8477-d9db05ef8dda service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:53:21.584991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323b75d-eb78-466a-8477-d9db05ef8dda service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:53:21.594948 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2879e4-1c5e-42c8-a617-7b446f2b2e2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:53:21.596230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2879e4-1c5e-42c8-a617-7b446f2b2e2f service swift] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:21.596861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2879e4-1c5e-42c8-a617-7b446f2b2e2f service swift] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:21.597524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2879e4-1c5e-42c8-a617-7b446f2b2e2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:21.614793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2879e4-1c5e-42c8-a617-7b446f2b2e2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:53:21.617475 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2879e4-1c5e-42c8-a617-7b446f2b2e2f service swift] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:53:21.633146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1113/2290] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:53:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:53:31.539895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=43 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:53:31.625849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=61 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:53:50.521179 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566aa9b-33ca-40c2-9fad-f5a92ac0c8d8 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:50.521933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566aa9b-33ca-40c2-9fad-f5a92ac0c8d8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:50.521933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566aa9b-33ca-40c2-9fad-f5a92ac0c8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:50.542377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f566aa9b-33ca-40c2-9fad-f5a92ac0c8d8 None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:53:50.543619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f566aa9b-33ca-40c2-9fad-f5a92ac0c8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:50.552798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1178/2291] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jul 19 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:53:50.568393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2879e4-1c5e-42c8-a617-7b446f2b2e2f service swift] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:53:50.581555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2879e4-1c5e-42c8-a617-7b446f2b2e2f service swift] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:53:50.587353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571ed29-ddc8-4c6f-b34f-3697323da58b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:53:50.588074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571ed29-ddc8-4c6f-b34f-3697323da58b service swift] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:50.588399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571ed29-ddc8-4c6f-b34f-3697323da58b service swift] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:50.588715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571ed29-ddc8-4c6f-b34f-3697323da58b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:50.599849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571ed29-ddc8-4c6f-b34f-3697323da58b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:53:50.601553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571ed29-ddc8-4c6f-b34f-3697323da58b service swift] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:53:50.611660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1114/2292] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:53:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 20:53:56.155161 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566aa9b-33ca-40c2-9fad-f5a92ac0c8d8 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:53:56.157001 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f566aa9b-33ca-40c2-9fad-f5a92ac0c8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:56.170640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566aa9b-33ca-40c2-9fad-f5a92ac0c8d8 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:53:56.174699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62148803-db12-43c7-8058-b2bcc53af988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:53:56.175680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62148803-db12-43c7-8058-b2bcc53af988 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:56.175978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62148803-db12-43c7-8058-b2bcc53af988 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:56.176260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62148803-db12-43c7-8058-b2bcc53af988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:56.178384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-62148803-db12-43c7-8058-b2bcc53af988 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:56.194104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62148803-db12-43c7-8058-b2bcc53af988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:53:56.195604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62148803-db12-43c7-8058-b2bcc53af988 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:53:56.208680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1179/2293] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:53:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:53:57.352314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571ed29-ddc8-4c6f-b34f-3697323da58b service swift] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:53:57.363577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571ed29-ddc8-4c6f-b34f-3697323da58b service swift] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:53:57.367391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08e2fc-5c97-4a56-af73-77c545d0f02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:53:57.368153 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08e2fc-5c97-4a56-af73-77c545d0f02d service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:53:57.368443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08e2fc-5c97-4a56-af73-77c545d0f02d service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:53:57.368720 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08e2fc-5c97-4a56-af73-77c545d0f02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:53:57.370670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ae08e2fc-5c97-4a56-af73-77c545d0f02d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:53:57.388137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08e2fc-5c97-4a56-af73-77c545d0f02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:53:57.390556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08e2fc-5c97-4a56-af73-77c545d0f02d service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:53:57.402035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1115/2294] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:53:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:54:00.225120 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75799a59-8eb7-4d38-be93-38bcd49fd7fd None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:00.226079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75799a59-8eb7-4d38-be93-38bcd49fd7fd None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:00.226079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75799a59-8eb7-4d38-be93-38bcd49fd7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:00.251581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-75799a59-8eb7-4d38-be93-38bcd49fd7fd None None] MFA Rules not processed for user `9fa96f51390e46e991c6e9afa385d2f7`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:00.253086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-75799a59-8eb7-4d38-be93-38bcd49fd7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:00.345188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1180/2295] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:54:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:54:07.397133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=12 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:10.334624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=34 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:42.488927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08e2fc-5c97-4a56-af73-77c545d0f02d service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:42.490419 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ae08e2fc-5c97-4a56-af73-77c545d0f02d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:42.507793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08e2fc-5c97-4a56-af73-77c545d0f02d service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:42.516658 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b34b92-aaea-4068-b079-3e3455350e1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:42.517391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b34b92-aaea-4068-b079-3e3455350e1b service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:42.517670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b34b92-aaea-4068-b079-3e3455350e1b service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:42.517942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b34b92-aaea-4068-b079-3e3455350e1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:42.519833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-82b34b92-aaea-4068-b079-3e3455350e1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:42.532364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b34b92-aaea-4068-b079-3e3455350e1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:42.534035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b34b92-aaea-4068-b079-3e3455350e1b service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:42.543659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1116/2296] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:46.137915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75799a59-8eb7-4d38-be93-38bcd49fd7fd None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:46.157941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75799a59-8eb7-4d38-be93-38bcd49fd7fd None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:46.168062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcba9da-3b7d-4892-89d2-6e3ff9635dcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:46.169283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcba9da-3b7d-4892-89d2-6e3ff9635dcc service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:46.172760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcba9da-3b7d-4892-89d2-6e3ff9635dcc service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:46.173332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcba9da-3b7d-4892-89d2-6e3ff9635dcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:46.176857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6bcba9da-3b7d-4892-89d2-6e3ff9635dcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:46.201956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcba9da-3b7d-4892-89d2-6e3ff9635dcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:46.204844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcba9da-3b7d-4892-89d2-6e3ff9635dcc service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:46.224295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1181/2297] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:52.539317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=6 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:56.216190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=6 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:58.594529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f77d-919d-49d4-bb85-20f1a3e980dd None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:58.595070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f77d-919d-49d4-bb85-20f1a3e980dd None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:58.595070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f77d-919d-49d4-bb85-20f1a3e980dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:58.615180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a4c4f77d-919d-49d4-bb85-20f1a3e980dd None None] MFA Rules not processed for user `af989ff6dc024ff19023e6f633915ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:58.616456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a4c4f77d-919d-49d4-bb85-20f1a3e980dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:58.623893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1117/2298] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:54:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:54:58.993521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcba9da-3b7d-4892-89d2-6e3ff9635dcc service nova] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:59.003895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcba9da-3b7d-4892-89d2-6e3ff9635dcc service nova] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:59.008985 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a94859-2f12-4f68-96a9-57caa6d86979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:59.009680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a94859-2f12-4f68-96a9-57caa6d86979 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:59.009959 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a94859-2f12-4f68-96a9-57caa6d86979 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:59.010235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a94859-2f12-4f68-96a9-57caa6d86979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:59.012041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-54a94859-2f12-4f68-96a9-57caa6d86979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:59.023758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a94859-2f12-4f68-96a9-57caa6d86979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:59.025332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a94859-2f12-4f68-96a9-57caa6d86979 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:59.036895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1182/2299] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:59.397161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4f77d-919d-49d4-bb85-20f1a3e980dd None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:59.399045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a4c4f77d-919d-49d4-bb85-20f1a3e980dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:59.424953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a94859-2f12-4f68-96a9-57caa6d86979 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:59.430447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4f77d-919d-49d4-bb85-20f1a3e980dd None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:59.440496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9700330-5c3c-42ed-96da-57174140dedc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:59.441499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a94859-2f12-4f68-96a9-57caa6d86979 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:59.441636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9700330-5c3c-42ed-96da-57174140dedc service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:59.441920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9700330-5c3c-42ed-96da-57174140dedc service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:59.442269 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9700330-5c3c-42ed-96da-57174140dedc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:59.444312 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-f9700330-5c3c-42ed-96da-57174140dedc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:59.446945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb36f91-e7c7-4ef3-97e9-5fa3c3fc271c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:59.447693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb36f91-e7c7-4ef3-97e9-5fa3c3fc271c service placement] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:59.447976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb36f91-e7c7-4ef3-97e9-5fa3c3fc271c service placement] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:59.448251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb36f91-e7c7-4ef3-97e9-5fa3c3fc271c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:59.450907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5cb36f91-e7c7-4ef3-97e9-5fa3c3fc271c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:59.467892 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb36f91-e7c7-4ef3-97e9-5fa3c3fc271c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:59.469575 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb36f91-e7c7-4ef3-97e9-5fa3c3fc271c service placement] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:59.484246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1183/2300] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:59.521612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9700330-5c3c-42ed-96da-57174140dedc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:59.523412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9700330-5c3c-42ed-96da-57174140dedc service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:59.535943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1118/2301] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:01.280784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb36f91-e7c7-4ef3-97e9-5fa3c3fc271c service placement] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:01.294040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb36f91-e7c7-4ef3-97e9-5fa3c3fc271c service placement] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:01.300661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0fed5-7207-4daa-85c6-af4ffde03565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:01.303579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0fed5-7207-4daa-85c6-af4ffde03565 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:01.304661 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0fed5-7207-4daa-85c6-af4ffde03565 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:01.305155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0fed5-7207-4daa-85c6-af4ffde03565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:01.309911 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-44e0fed5-7207-4daa-85c6-af4ffde03565 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:01.344201 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0fed5-7207-4daa-85c6-af4ffde03565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:01.347305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0fed5-7207-4daa-85c6-af4ffde03565 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:01.364906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1184/2302] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:09.530531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=31 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:11.357055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=18 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:12.440122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9700330-5c3c-42ed-96da-57174140dedc service nova] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:12.456000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9700330-5c3c-42ed-96da-57174140dedc service nova] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:12.464620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b091ba-8979-4394-861d-b3e158da90a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:12.465856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b091ba-8979-4394-861d-b3e158da90a7 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:12.466377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b091ba-8979-4394-861d-b3e158da90a7 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:12.466879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b091ba-8979-4394-861d-b3e158da90a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:12.471483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-12b091ba-8979-4394-861d-b3e158da90a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:12.486595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b091ba-8979-4394-861d-b3e158da90a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:12.488582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b091ba-8979-4394-861d-b3e158da90a7 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:12.502857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1119/2303] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:12.903724 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5f15d-8164-4fd0-95ba-f18d90173dea None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:12.904927 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5f15d-8164-4fd0-95ba-f18d90173dea None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:12.905255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5f15d-8164-4fd0-95ba-f18d90173dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:12.936400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-00b5f15d-8164-4fd0-95ba-f18d90173dea None None] MFA Rules not processed for user `31790576f8734f75adbbd8e9c0d61721`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:12.938268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-00b5f15d-8164-4fd0-95ba-f18d90173dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:12.949381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1185/2304] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:13.391341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b091ba-8979-4394-861d-b3e158da90a7 service nova] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:13.401814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b091ba-8979-4394-861d-b3e158da90a7 service nova] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:13.406567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bf4ae-b738-4c2d-b203-3618722b7232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:13.407272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bf4ae-b738-4c2d-b203-3618722b7232 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:13.407604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bf4ae-b738-4c2d-b203-3618722b7232 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:13.407914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bf4ae-b738-4c2d-b203-3618722b7232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:13.410205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-9f6bf4ae-b738-4c2d-b203-3618722b7232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:13.422375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bf4ae-b738-4c2d-b203-3618722b7232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:13.424067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bf4ae-b738-4c2d-b203-3618722b7232 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:13.434057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1120/2305] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:55:13.904920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5f15d-8164-4fd0-95ba-f18d90173dea None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:13.906645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-00b5f15d-8164-4fd0-95ba-f18d90173dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:14.020490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5f15d-8164-4fd0-95ba-f18d90173dea None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:14.027614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99f71d-e07c-4d73-af78-9dcc652c1001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:14.029023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99f71d-e07c-4d73-af78-9dcc652c1001 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:14.030080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99f71d-e07c-4d73-af78-9dcc652c1001 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:14.030574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99f71d-e07c-4d73-af78-9dcc652c1001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:14.033780 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2d99f71d-e07c-4d73-af78-9dcc652c1001 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:14.055257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99f71d-e07c-4d73-af78-9dcc652c1001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:14.058609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99f71d-e07c-4d73-af78-9dcc652c1001 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:14.081458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1186/2306] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:55:23.429755 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=12 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:24.068356 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=29 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:32.745546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bf4ae-b738-4c2d-b203-3618722b7232 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:32.762087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bf4ae-b738-4c2d-b203-3618722b7232 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:32.772544 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e7649-59a8-4d10-be4c-53b1da918a60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:32.773624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7649-59a8-4d10-be4c-53b1da918a60 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:32.773908 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7649-59a8-4d10-be4c-53b1da918a60 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:32.774190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7649-59a8-4d10-be4c-53b1da918a60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:32.776736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-325e7649-59a8-4d10-be4c-53b1da918a60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:32.790919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e7649-59a8-4d10-be4c-53b1da918a60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:32.792029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e7649-59a8-4d10-be4c-53b1da918a60 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:32.804168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1121/2307] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:36.988636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99f71d-e07c-4d73-af78-9dcc652c1001 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:36.998539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99f71d-e07c-4d73-af78-9dcc652c1001 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:37.002697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14032c-cb33-4998-a06a-4ad1a68dd8ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:37.003345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14032c-cb33-4998-a06a-4ad1a68dd8ea service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:37.003630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14032c-cb33-4998-a06a-4ad1a68dd8ea service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:37.003915 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14032c-cb33-4998-a06a-4ad1a68dd8ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:37.005766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5b14032c-cb33-4998-a06a-4ad1a68dd8ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:37.075495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14032c-cb33-4998-a06a-4ad1a68dd8ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:37.076928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14032c-cb33-4998-a06a-4ad1a68dd8ea service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:37.087397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1187/2308] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:55:40.448539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e7649-59a8-4d10-be4c-53b1da918a60 service nova] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:40.450388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-325e7649-59a8-4d10-be4c-53b1da918a60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:40.467882 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e7649-59a8-4d10-be4c-53b1da918a60 service nova] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:40.473426 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acf687-b079-4240-8ef7-134ecf5117b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:40.474106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acf687-b079-4240-8ef7-134ecf5117b6 service placement] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:40.474392 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acf687-b079-4240-8ef7-134ecf5117b6 service placement] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:40.474671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acf687-b079-4240-8ef7-134ecf5117b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:40.476607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e5acf687-b079-4240-8ef7-134ecf5117b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:40.492686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5acf687-b079-4240-8ef7-134ecf5117b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:40.495113 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5acf687-b079-4240-8ef7-134ecf5117b6 service placement] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:40.514051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1122/2309] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:40.560758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14032c-cb33-4998-a06a-4ad1a68dd8ea service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:40.562271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5b14032c-cb33-4998-a06a-4ad1a68dd8ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:40.576009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14032c-cb33-4998-a06a-4ad1a68dd8ea service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:40.581416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363cccb-1b9f-4300-9fdd-673bd1536910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:40.582313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363cccb-1b9f-4300-9fdd-673bd1536910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:40.582599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363cccb-1b9f-4300-9fdd-673bd1536910 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:40.582880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363cccb-1b9f-4300-9fdd-673bd1536910 admin admin] PATH_INFO: `/v3/users/9fa96f51390e46e991c6e9afa385d2f7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:40.584150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363cccb-1b9f-4300-9fdd-673bd1536910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fa96f51390e46e991c6e9afa385d2f7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:40.586832 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363cccb-1b9f-4300-9fdd-673bd1536910 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:40.655212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8363cccb-1b9f-4300-9fdd-673bd1536910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fa96f51390e46e991c6e9afa385d2f7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:40.668218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8363cccb-1b9f-4300-9fdd-673bd1536910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fa96f51390e46e991c6e9afa385d2f7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:40.678104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8363cccb-1b9f-4300-9fdd-673bd1536910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fa96f51390e46e991c6e9afa385d2f7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:40.678104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8363cccb-1b9f-4300-9fdd-673bd1536910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fa96f51390e46e991c6e9afa385d2f7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:40.684783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8363cccb-1b9f-4300-9fdd-673bd1536910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fa96f51390e46e991c6e9afa385d2f7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:40.691478 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1188/2310] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:40 2022] DELETE /identity/v3/users/9fa96f51390e46e991c6e9afa385d2f7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:40.882218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acf687-b079-4240-8ef7-134ecf5117b6 service placement] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:40.900622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acf687-b079-4240-8ef7-134ecf5117b6 service placement] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:40.965268 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:40.966181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:40.966462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:40.966737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] PATH_INFO: `/v3/projects/6cc474e26dd8444b99645279c4d133bf` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:40.968514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc474e26dd8444b99645279c4d133bf)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:40.970747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:41.026971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc474e26dd8444b99645279c4d133bf', 'request_id': 'req-2bc877dd-6ea8-4842-b3c6-439627345110'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:41.038602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc474e26dd8444b99645279c4d133bf', 'request_id': 'req-2bc877dd-6ea8-4842-b3c6-439627345110'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:41.044561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc474e26dd8444b99645279c4d133bf', 'request_id': 'req-2bc877dd-6ea8-4842-b3c6-439627345110'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:41.058678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] The token cache is being invalidate because project 6cc474e26dd8444b99645279c4d133bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:41.058879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:41.059665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2bc877dd-6ea8-4842-b3c6-439627345110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:41.061283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1123/2311] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:40 2022] DELETE /identity/v3/projects/6cc474e26dd8444b99645279c4d133bf => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:41.066326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0347ec7-22f8-42df-96ff-3df3b998a75f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.066645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0347ec7-22f8-42df-96ff-3df3b998a75f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.066922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0347ec7-22f8-42df-96ff-3df3b998a75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.085873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-c0347ec7-22f8-42df-96ff-3df3b998a75f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:41.087097 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c0347ec7-22f8-42df-96ff-3df3b998a75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:41.154120 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1189/2312] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:41.164901 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d73f1-2ce1-40b1-ae7f-0bbbfc8e3a29 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.165560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d73f1-2ce1-40b1-ae7f-0bbbfc8e3a29 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.166042 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d73f1-2ce1-40b1-ae7f-0bbbfc8e3a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.189902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-157d73f1-2ce1-40b1-ae7f-0bbbfc8e3a29 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:41.191138 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-157d73f1-2ce1-40b1-ae7f-0bbbfc8e3a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:41.199178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1124/2313] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:41.204575 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0347ec7-22f8-42df-96ff-3df3b998a75f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:41.214710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0347ec7-22f8-42df-96ff-3df3b998a75f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:41.220471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf82af-081b-4022-a4ef-f7d5ae346bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:41.221174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf82af-081b-4022-a4ef-f7d5ae346bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.221499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf82af-081b-4022-a4ef-f7d5ae346bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.221779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf82af-081b-4022-a4ef-f7d5ae346bb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.222520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf82af-081b-4022-a4ef-f7d5ae346bb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:41.222829 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf82af-081b-4022-a4ef-f7d5ae346bb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:41.224456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf82af-081b-4022-a4ef-f7d5ae346bb5 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:41.226246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:41.241561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1190/2314] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:55:41.243984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d73f1-2ce1-40b1-ae7f-0bbbfc8e3a29 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:41.253582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d73f1-2ce1-40b1-ae7f-0bbbfc8e3a29 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:41.259395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7281b2-060d-4b9c-bc8b-e654dd4bf057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:41.259395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7281b2-060d-4b9c-bc8b-e654dd4bf057 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.259658 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7281b2-060d-4b9c-bc8b-e654dd4bf057 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.259833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7281b2-060d-4b9c-bc8b-e654dd4bf057 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.260694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7281b2-060d-4b9c-bc8b-e654dd4bf057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:41.262652 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7281b2-060d-4b9c-bc8b-e654dd4bf057 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:41.300337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1125/2315] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:41 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:41.304587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf82af-081b-4022-a4ef-f7d5ae346bb5 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:41.316574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf82af-081b-4022-a4ef-f7d5ae346bb5 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:41.388347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f136a-9d37-4bc4-8a6f-7297d3e7d9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:41.389708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f136a-9d37-4bc4-8a6f-7297d3e7d9eb admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.389995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f136a-9d37-4bc4-8a6f-7297d3e7d9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.390273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f136a-9d37-4bc4-8a6f-7297d3e7d9eb admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.391117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f136a-9d37-4bc4-8a6f-7297d3e7d9eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:41.393691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f136a-9d37-4bc4-8a6f-7297d3e7d9eb admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:41.413380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1191/2316] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:41 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:41.417430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7281b2-060d-4b9c-bc8b-e654dd4bf057 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:41.427826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7281b2-060d-4b9c-bc8b-e654dd4bf057 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:41.432463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a84c94-58ed-407a-ab47-8a0281bd2c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:41.433128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a84c94-58ed-407a-ab47-8a0281bd2c82 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.433447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a84c94-58ed-407a-ab47-8a0281bd2c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.433729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a84c94-58ed-407a-ab47-8a0281bd2c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.434466 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a84c94-58ed-407a-ab47-8a0281bd2c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:41.436575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a84c94-58ed-407a-ab47-8a0281bd2c82 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:41.449001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1126/2317] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:41.454565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f136a-9d37-4bc4-8a6f-7297d3e7d9eb admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:41.465412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f136a-9d37-4bc4-8a6f-7297d3e7d9eb admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:41.470299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0eccef-3866-4efb-8cfe-a8b610bf3a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:41.471114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eccef-3866-4efb-8cfe-a8b610bf3a41 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.471398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eccef-3866-4efb-8cfe-a8b610bf3a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.471680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eccef-3866-4efb-8cfe-a8b610bf3a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.472415 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0eccef-3866-4efb-8cfe-a8b610bf3a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:41.474061 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0eccef-3866-4efb-8cfe-a8b610bf3a41 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:41.484952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1192/2318] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:41.489495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a84c94-58ed-407a-ab47-8a0281bd2c82 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:41.500180 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a84c94-58ed-407a-ab47-8a0281bd2c82 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:41.504866 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d715f5-b75e-48b0-b937-bc3a2b061913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:41.505771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d715f5-b75e-48b0-b937-bc3a2b061913 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.506058 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d715f5-b75e-48b0-b937-bc3a2b061913 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.506339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d715f5-b75e-48b0-b937-bc3a2b061913 admin admin] PATH_INFO: `/v3/projects/5ee60f42cc4342aeae135c849b932c5f/users/95c16b9370d346148a9ab7cd60adc11d/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.522742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d715f5-b75e-48b0-b937-bc3a2b061913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ee60f42cc4342aeae135c849b932c5f, user_id=95c16b9370d346148a9ab7cd60adc11d, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:41.525719 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d715f5-b75e-48b0-b937-bc3a2b061913 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:41.536377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:41.542083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1127/2319] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:41 2022] PUT /identity/v3/projects/5ee60f42cc4342aeae135c849b932c5f/users/95c16b9370d346148a9ab7cd60adc11d/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:41.553532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43d4dc-356d-4dd3-869f-af94fc7ea1a6 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.553804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43d4dc-356d-4dd3-869f-af94fc7ea1a6 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.554089 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43d4dc-356d-4dd3-869f-af94fc7ea1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.582583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-8a43d4dc-356d-4dd3-869f-af94fc7ea1a6 None None] MFA Rules not processed for user `95c16b9370d346148a9ab7cd60adc11d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:41.583703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8a43d4dc-356d-4dd3-869f-af94fc7ea1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:41.652444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1193/2320] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:41.657262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d715f5-b75e-48b0-b937-bc3a2b061913 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:41.720550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d715f5-b75e-48b0-b937-bc3a2b061913 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:41.725181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e144ccf-8cf3-4aad-b509-30a168fb2960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:41.725873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e144ccf-8cf3-4aad-b509-30a168fb2960 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.726151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e144ccf-8cf3-4aad-b509-30a168fb2960 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.726424 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e144ccf-8cf3-4aad-b509-30a168fb2960 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.727375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e144ccf-8cf3-4aad-b509-30a168fb2960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:41.729240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e144ccf-8cf3-4aad-b509-30a168fb2960 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:41.762121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1128/2321] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:41 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:41.767256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43d4dc-356d-4dd3-869f-af94fc7ea1a6 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:41.779197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43d4dc-356d-4dd3-869f-af94fc7ea1a6 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:41.847372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d20f0-bf66-423a-9351-072b85626f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:41.847962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d20f0-bf66-423a-9351-072b85626f7a admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.848242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d20f0-bf66-423a-9351-072b85626f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.848519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d20f0-bf66-423a-9351-072b85626f7a admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.849225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d20f0-bf66-423a-9351-072b85626f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:41.851183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d20f0-bf66-423a-9351-072b85626f7a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:41.859814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:41.872853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1194/2322] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:41 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:55:41.877482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e144ccf-8cf3-4aad-b509-30a168fb2960 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:41.888752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e144ccf-8cf3-4aad-b509-30a168fb2960 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:41.898621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fef942-db7e-4eee-8301-6fec0d1a5f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:41.899944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fef942-db7e-4eee-8301-6fec0d1a5f81 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.900336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fef942-db7e-4eee-8301-6fec0d1a5f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.900678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fef942-db7e-4eee-8301-6fec0d1a5f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.901914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fef942-db7e-4eee-8301-6fec0d1a5f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:41.905907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fef942-db7e-4eee-8301-6fec0d1a5f81 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:41.926605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1129/2323] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:41.931032 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d20f0-bf66-423a-9351-072b85626f7a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:41.942873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d20f0-bf66-423a-9351-072b85626f7a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:41.947844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f22c54-ccab-4ef8-92aa-e114efb284a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:41.948748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f22c54-ccab-4ef8-92aa-e114efb284a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:41.949027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f22c54-ccab-4ef8-92aa-e114efb284a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:41.949298 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f22c54-ccab-4ef8-92aa-e114efb284a5 admin admin] PATH_INFO: `/v3/projects/f81286e7c1bb42cdb9004396defd9897/users/6299adffa31b46ea924847fbda930631/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:41.966355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f22c54-ccab-4ef8-92aa-e114efb284a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81286e7c1bb42cdb9004396defd9897, user_id=6299adffa31b46ea924847fbda930631, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:41.969066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f22c54-ccab-4ef8-92aa-e114efb284a5 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:41.983955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1195/2324] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:41 2022] PUT /identity/v3/projects/f81286e7c1bb42cdb9004396defd9897/users/6299adffa31b46ea924847fbda930631/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:41.988443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fef942-db7e-4eee-8301-6fec0d1a5f81 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:41.999734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fef942-db7e-4eee-8301-6fec0d1a5f81 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:42.062855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a536dc-cd9b-4c30-b98d-0352a4c3b3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:42.063521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a536dc-cd9b-4c30-b98d-0352a4c3b3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:42.063805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a536dc-cd9b-4c30-b98d-0352a4c3b3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:42.064081 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a536dc-cd9b-4c30-b98d-0352a4c3b3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:42.064804 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a536dc-cd9b-4c30-b98d-0352a4c3b3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:42.066592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a536dc-cd9b-4c30-b98d-0352a4c3b3f9 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:42.076938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1130/2325] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:55:42.081735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f22c54-ccab-4ef8-92aa-e114efb284a5 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:42.094335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f22c54-ccab-4ef8-92aa-e114efb284a5 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:42.099130 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43fdb3-9014-4cd6-b443-00f7bb80b9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:42.099971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43fdb3-9014-4cd6-b443-00f7bb80b9ad admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:42.100233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43fdb3-9014-4cd6-b443-00f7bb80b9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:42.100506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43fdb3-9014-4cd6-b443-00f7bb80b9ad admin admin] PATH_INFO: `/v3/projects/f81286e7c1bb42cdb9004396defd9897/users/6299adffa31b46ea924847fbda930631/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:42.102768 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43fdb3-9014-4cd6-b443-00f7bb80b9ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81286e7c1bb42cdb9004396defd9897, user_id=6299adffa31b46ea924847fbda930631, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:42.105468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f43fdb3-9014-4cd6-b443-00f7bb80b9ad admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:42.120340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1196/2326] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:42 2022] PUT /identity/v3/projects/f81286e7c1bb42cdb9004396defd9897/users/6299adffa31b46ea924847fbda930631/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:42.130926 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe3dbd-7646-49b9-81f4-5019b776c28a None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:42.131213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe3dbd-7646-49b9-81f4-5019b776c28a None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:42.131490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe3dbd-7646-49b9-81f4-5019b776c28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:42.165205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-42fe3dbd-7646-49b9-81f4-5019b776c28a None None] MFA Rules not processed for user `6299adffa31b46ea924847fbda930631`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:42.166491 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-42fe3dbd-7646-49b9-81f4-5019b776c28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:42.234230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:42.244315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1131/2327] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:42.255964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43fdb3-9014-4cd6-b443-00f7bb80b9ad admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:42.258032 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2f43fdb3-9014-4cd6-b443-00f7bb80b9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:42.318432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43fdb3-9014-4cd6-b443-00f7bb80b9ad admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:42.322055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf5919-c749-4b9b-ba70-59a875fa49cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:42.322701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf5919-c749-4b9b-ba70-59a875fa49cb service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:42.322976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf5919-c749-4b9b-ba70-59a875fa49cb service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:42.323269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf5919-c749-4b9b-ba70-59a875fa49cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:42.332304 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf5919-c749-4b9b-ba70-59a875fa49cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:42.333781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf5919-c749-4b9b-ba70-59a875fa49cb service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:42.344869 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1197/2328] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:42.415285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe3dbd-7646-49b9-81f4-5019b776c28a None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:42.508097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe3dbd-7646-49b9-81f4-5019b776c28a None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:42.515724 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:42.517182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:42.517733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:42.518214 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] PATH_INFO: `/v3/users/95c16b9370d346148a9ab7cd60adc11d` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:42.520385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c16b9370d346148a9ab7cd60adc11d)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:42.523897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:42.588987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c16b9370d346148a9ab7cd60adc11d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:42.596521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c16b9370d346148a9ab7cd60adc11d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:42.605547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c16b9370d346148a9ab7cd60adc11d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:42.619179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c16b9370d346148a9ab7cd60adc11d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:42.619918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c16b9370d346148a9ab7cd60adc11d'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:42.626430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1132/2329] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:42 2022] DELETE /identity/v3/users/95c16b9370d346148a9ab7cd60adc11d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:42.630949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10dfc3-93f8-47a5-b7a6-d4c24bfbe895 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:42.631441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10dfc3-93f8-47a5-b7a6-d4c24bfbe895 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:42.631554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10dfc3-93f8-47a5-b7a6-d4c24bfbe895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:42.649921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-dc10dfc3-93f8-47a5-b7a6-d4c24bfbe895 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:42.651144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-dc10dfc3-93f8-47a5-b7a6-d4c24bfbe895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:42.716399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1198/2330] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:42.728530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:42.730047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:42.790950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68ce92-c26f-4aaf-9704-2ebebad516c6 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:42.794999 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab6549-a068-430a-bece-53ab7da44067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:42.796246 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab6549-a068-430a-bece-53ab7da44067 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:42.796539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab6549-a068-430a-bece-53ab7da44067 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:42.796799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab6549-a068-430a-bece-53ab7da44067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:42.805830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab6549-a068-430a-bece-53ab7da44067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:42.807250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab6549-a068-430a-bece-53ab7da44067 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:42.816897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1133/2331] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:43.104870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10dfc3-93f8-47a5-b7a6-d4c24bfbe895 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:43.106553 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-dc10dfc3-93f8-47a5-b7a6-d4c24bfbe895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:43.129541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10dfc3-93f8-47a5-b7a6-d4c24bfbe895 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:43.136920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:43.138344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:43.138686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:43.139045 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] PATH_INFO: `/v3/projects/5ee60f42cc4342aeae135c849b932c5f` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:43.140711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ee60f42cc4342aeae135c849b932c5f)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:43.143464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:43.197618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ee60f42cc4342aeae135c849b932c5f', 'request_id': 'req-23bf7527-fe29-4c6e-ad02-02d32483de6d'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.200921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:43.209568 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ee60f42cc4342aeae135c849b932c5f', 'request_id': 'req-23bf7527-fe29-4c6e-ad02-02d32483de6d'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.220746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ee60f42cc4342aeae135c849b932c5f', 'request_id': 'req-23bf7527-fe29-4c6e-ad02-02d32483de6d'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.240384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] The token cache is being invalidate because project 5ee60f42cc4342aeae135c849b932c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:43.240852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.241994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.243740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1199/2332] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:43 2022] DELETE /identity/v3/projects/5ee60f42cc4342aeae135c849b932c5f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:43.247963 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab6549-a068-430a-bece-53ab7da44067 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:43.258470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab6549-a068-430a-bece-53ab7da44067 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:43.272459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:43.320795 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc48461-cc08-40ff-96f9-35913bafa1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:43.321884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc48461-cc08-40ff-96f9-35913bafa1a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:43.322220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc48461-cc08-40ff-96f9-35913bafa1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:43.322533 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc48461-cc08-40ff-96f9-35913bafa1a3 admin admin] PATH_INFO: `/v3/users/6299adffa31b46ea924847fbda930631` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:43.323777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc48461-cc08-40ff-96f9-35913bafa1a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6299adffa31b46ea924847fbda930631)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:43.325752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc48461-cc08-40ff-96f9-35913bafa1a3 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:43.383553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-dbc48461-cc08-40ff-96f9-35913bafa1a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6299adffa31b46ea924847fbda930631'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.391603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-dbc48461-cc08-40ff-96f9-35913bafa1a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6299adffa31b46ea924847fbda930631'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.399035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-dbc48461-cc08-40ff-96f9-35913bafa1a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6299adffa31b46ea924847fbda930631'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.411687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-dbc48461-cc08-40ff-96f9-35913bafa1a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6299adffa31b46ea924847fbda930631'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.412377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-dbc48461-cc08-40ff-96f9-35913bafa1a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6299adffa31b46ea924847fbda930631'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.418577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1134/2333] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:43 2022] DELETE /identity/v3/users/6299adffa31b46ea924847fbda930631 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:43.703784 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:43.705216 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:43.731792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf7527-fe29-4c6e-ad02-02d32483de6d admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:43.791302 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:43.792115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:43.792399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:43.792697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] PATH_INFO: `/v3/projects/f81286e7c1bb42cdb9004396defd9897` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:43.793947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f81286e7c1bb42cdb9004396defd9897)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:43.795985 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:43.861401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81286e7c1bb42cdb9004396defd9897', 'request_id': 'req-5cdc378c-2874-4d6a-87a5-4413daeaf028'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.869725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81286e7c1bb42cdb9004396defd9897', 'request_id': 'req-5cdc378c-2874-4d6a-87a5-4413daeaf028'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.876555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81286e7c1bb42cdb9004396defd9897', 'request_id': 'req-5cdc378c-2874-4d6a-87a5-4413daeaf028'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.888110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] The token cache is being invalidate because project f81286e7c1bb42cdb9004396defd9897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:43.888411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.889882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5cdc378c-2874-4d6a-87a5-4413daeaf028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:43.891672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1200/2334] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:43 2022] DELETE /identity/v3/projects/f81286e7c1bb42cdb9004396defd9897 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:43.897263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005f68b-af2c-4812-8aed-d75c2fd65d95 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:43.897812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005f68b-af2c-4812-8aed-d75c2fd65d95 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:43.897997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005f68b-af2c-4812-8aed-d75c2fd65d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:43.918684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-e005f68b-af2c-4812-8aed-d75c2fd65d95 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:43.919927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e005f68b-af2c-4812-8aed-d75c2fd65d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:43.981528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1135/2335] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:43.990079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07a9f-1ea1-4bcb-ad65-c38fd2947e0b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:43.990402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07a9f-1ea1-4bcb-ad65-c38fd2947e0b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:43.990684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07a9f-1ea1-4bcb-ad65-c38fd2947e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.011002 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-62a07a9f-1ea1-4bcb-ad65-c38fd2947e0b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:44.012235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-62a07a9f-1ea1-4bcb-ad65-c38fd2947e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:44.020423 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1201/2336] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:44.025241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005f68b-af2c-4812-8aed-d75c2fd65d95 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.037071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005f68b-af2c-4812-8aed-d75c2fd65d95 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.041754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750a8c8-aebd-4046-ba5c-14df1252f41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.042359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750a8c8-aebd-4046-ba5c-14df1252f41b admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.042639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750a8c8-aebd-4046-ba5c-14df1252f41b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.042912 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750a8c8-aebd-4046-ba5c-14df1252f41b admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.043638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750a8c8-aebd-4046-ba5c-14df1252f41b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:44.043940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750a8c8-aebd-4046-ba5c-14df1252f41b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.046054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750a8c8-aebd-4046-ba5c-14df1252f41b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.058511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1136/2337] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:55:44.063934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a07a9f-1ea1-4bcb-ad65-c38fd2947e0b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.079614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a07a9f-1ea1-4bcb-ad65-c38fd2947e0b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.084643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4110b-688f-4303-ae52-3f32b82b6631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.085327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4110b-688f-4303-ae52-3f32b82b6631 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.085650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4110b-688f-4303-ae52-3f32b82b6631 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.085934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4110b-688f-4303-ae52-3f32b82b6631 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.086657 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4110b-688f-4303-ae52-3f32b82b6631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.088591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4110b-688f-4303-ae52-3f32b82b6631 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.117596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1202/2338] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:44 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:44.121903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750a8c8-aebd-4046-ba5c-14df1252f41b admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.134190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750a8c8-aebd-4046-ba5c-14df1252f41b admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.157953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:44.197975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e50e27-a11d-4d16-a09f-1f8d7940b8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.198671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e50e27-a11d-4d16-a09f-1f8d7940b8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.198957 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e50e27-a11d-4d16-a09f-1f8d7940b8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.199236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e50e27-a11d-4d16-a09f-1f8d7940b8d4 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.199921 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e50e27-a11d-4d16-a09f-1f8d7940b8d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.201906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e50e27-a11d-4d16-a09f-1f8d7940b8d4 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.222903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1137/2339] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:44 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:44.227271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4110b-688f-4303-ae52-3f32b82b6631 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.238919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4110b-688f-4303-ae52-3f32b82b6631 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.243757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da3f2b-f61b-4828-b81d-e14299c09a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.244429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da3f2b-f61b-4828-b81d-e14299c09a09 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.244715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da3f2b-f61b-4828-b81d-e14299c09a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.244995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da3f2b-f61b-4828-b81d-e14299c09a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.245771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da3f2b-f61b-4828-b81d-e14299c09a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.248000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da3f2b-f61b-4828-b81d-e14299c09a09 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.260035 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1203/2340] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:44.263874 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e50e27-a11d-4d16-a09f-1f8d7940b8d4 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.273977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e50e27-a11d-4d16-a09f-1f8d7940b8d4 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.278743 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2f702-a138-48f9-875b-f62a2cbdf7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.279994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2f702-a138-48f9-875b-f62a2cbdf7fd admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.280296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2f702-a138-48f9-875b-f62a2cbdf7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.280571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2f702-a138-48f9-875b-f62a2cbdf7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.281301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2f702-a138-48f9-875b-f62a2cbdf7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.282939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2f702-a138-48f9-875b-f62a2cbdf7fd admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.293939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1138/2341] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:44.298698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da3f2b-f61b-4828-b81d-e14299c09a09 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.309270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da3f2b-f61b-4828-b81d-e14299c09a09 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.314448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092c6e5-3c35-4430-91e9-b092564d1cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.315354 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092c6e5-3c35-4430-91e9-b092564d1cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.315641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092c6e5-3c35-4430-91e9-b092564d1cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.315918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092c6e5-3c35-4430-91e9-b092564d1cf4 admin admin] PATH_INFO: `/v3/projects/ae27e18c8a5c406eab5501467e563444/users/4eec9efdcafc4afebf8bd047d8d205f3/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.332255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092c6e5-3c35-4430-91e9-b092564d1cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae27e18c8a5c406eab5501467e563444, user_id=4eec9efdcafc4afebf8bd047d8d205f3, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.334976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092c6e5-3c35-4430-91e9-b092564d1cf4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.349237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1204/2342] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:44 2022] PUT /identity/v3/projects/ae27e18c8a5c406eab5501467e563444/users/4eec9efdcafc4afebf8bd047d8d205f3/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:44.363083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a7b62-1d5b-4958-b5b7-ca9db644d54b None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.363511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a7b62-1d5b-4958-b5b7-ca9db644d54b None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.363910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a7b62-1d5b-4958-b5b7-ca9db644d54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.406663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-686a7b62-1d5b-4958-b5b7-ca9db644d54b None None] MFA Rules not processed for user `4eec9efdcafc4afebf8bd047d8d205f3`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:44.407915 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-686a7b62-1d5b-4958-b5b7-ca9db644d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:44.480356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1139/2343] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:44.485781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092c6e5-3c35-4430-91e9-b092564d1cf4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.495728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:44.556144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092c6e5-3c35-4430-91e9-b092564d1cf4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.561262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d74e2-baab-4a28-8443-09564cd21d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.561955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d74e2-baab-4a28-8443-09564cd21d63 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.562239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d74e2-baab-4a28-8443-09564cd21d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.562508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d74e2-baab-4a28-8443-09564cd21d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.563216 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d74e2-baab-4a28-8443-09564cd21d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.565102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d74e2-baab-4a28-8443-09564cd21d63 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.591766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1205/2344] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:44 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:44.604401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a7b62-1d5b-4958-b5b7-ca9db644d54b None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.616439 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a7b62-1d5b-4958-b5b7-ca9db644d54b None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.675089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd70e1-158b-4739-93ad-c4d8089cf3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.675774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd70e1-158b-4739-93ad-c4d8089cf3fa admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.676062 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd70e1-158b-4739-93ad-c4d8089cf3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.676342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd70e1-158b-4739-93ad-c4d8089cf3fa admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.677179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd70e1-158b-4739-93ad-c4d8089cf3fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.679163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd70e1-158b-4739-93ad-c4d8089cf3fa admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.709516 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1140/2345] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:44 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:44.714058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d74e2-baab-4a28-8443-09564cd21d63 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.725068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d74e2-baab-4a28-8443-09564cd21d63 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.729947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5649a-7e1c-40cd-8f40-79e9acaf22ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.730608 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5649a-7e1c-40cd-8f40-79e9acaf22ac admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.730893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5649a-7e1c-40cd-8f40-79e9acaf22ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.731169 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5649a-7e1c-40cd-8f40-79e9acaf22ac admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.731922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5649a-7e1c-40cd-8f40-79e9acaf22ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.734841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5649a-7e1c-40cd-8f40-79e9acaf22ac admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.745872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1206/2346] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:44.750670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd70e1-158b-4739-93ad-c4d8089cf3fa admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.760975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd70e1-158b-4739-93ad-c4d8089cf3fa admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.767150 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0b830-f22e-488e-8dc9-5d237c704145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.768048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0b830-f22e-488e-8dc9-5d237c704145 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.768332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0b830-f22e-488e-8dc9-5d237c704145 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.768608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0b830-f22e-488e-8dc9-5d237c704145 admin admin] PATH_INFO: `/v3/projects/616a57536e624f53a127f07dc9415713/users/c89699e3a4ec40e59d8ec0e2176c4fa8/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.793245 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0b830-f22e-488e-8dc9-5d237c704145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=616a57536e624f53a127f07dc9415713, user_id=c89699e3a4ec40e59d8ec0e2176c4fa8, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.799002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0b830-f22e-488e-8dc9-5d237c704145 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.811968 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:44.818575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1141/2347] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:44 2022] PUT /identity/v3/projects/616a57536e624f53a127f07dc9415713/users/c89699e3a4ec40e59d8ec0e2176c4fa8/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:44.822650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5649a-7e1c-40cd-8f40-79e9acaf22ac admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.832622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5649a-7e1c-40cd-8f40-79e9acaf22ac admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.889616 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1402af-85f7-4eed-9df9-a272f9f8bbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.890492 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1402af-85f7-4eed-9df9-a272f9f8bbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.890627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1402af-85f7-4eed-9df9-a272f9f8bbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.890863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1402af-85f7-4eed-9df9-a272f9f8bbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.891625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1402af-85f7-4eed-9df9-a272f9f8bbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.893224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1402af-85f7-4eed-9df9-a272f9f8bbd2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.903903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1207/2348] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:44.909925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0b830-f22e-488e-8dc9-5d237c704145 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.921416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0b830-f22e-488e-8dc9-5d237c704145 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.926202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0530a-cf90-47fb-b05f-b9cab5f9f4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.927091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0530a-cf90-47fb-b05f-b9cab5f9f4c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.927372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0530a-cf90-47fb-b05f-b9cab5f9f4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.927655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0530a-cf90-47fb-b05f-b9cab5f9f4c5 admin admin] PATH_INFO: `/v3/projects/616a57536e624f53a127f07dc9415713/users/c89699e3a4ec40e59d8ec0e2176c4fa8/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.930319 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0530a-cf90-47fb-b05f-b9cab5f9f4c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=616a57536e624f53a127f07dc9415713, user_id=c89699e3a4ec40e59d8ec0e2176c4fa8, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.932984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0530a-cf90-47fb-b05f-b9cab5f9f4c5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.944677 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1142/2349] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:44 2022] PUT /identity/v3/projects/616a57536e624f53a127f07dc9415713/users/c89699e3a4ec40e59d8ec0e2176c4fa8/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:44.956178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ea289-312a-40b4-8bc7-4ac1d5a79700 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.956466 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ea289-312a-40b4-8bc7-4ac1d5a79700 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.956744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ea289-312a-40b4-8bc7-4ac1d5a79700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.985011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-959ea289-312a-40b4-8bc7-4ac1d5a79700 None None] MFA Rules not processed for user `c89699e3a4ec40e59d8ec0e2176c4fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:44.986275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-959ea289-312a-40b4-8bc7-4ac1d5a79700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:45.057811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1208/2350] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:45.070919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0530a-cf90-47fb-b05f-b9cab5f9f4c5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:45.073503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e6b0530a-cf90-47fb-b05f-b9cab5f9f4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:45.139084 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0530a-cf90-47fb-b05f-b9cab5f9f4c5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:45.142845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa553ebb-af33-4ebe-9de3-1f8fedef50f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:45.143493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa553ebb-af33-4ebe-9de3-1f8fedef50f4 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:45.143781 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa553ebb-af33-4ebe-9de3-1f8fedef50f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:45.144060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa553ebb-af33-4ebe-9de3-1f8fedef50f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:45.154011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa553ebb-af33-4ebe-9de3-1f8fedef50f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:45.156006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa553ebb-af33-4ebe-9de3-1f8fedef50f4 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:45.167658 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1143/2351] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:45.488589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ea289-312a-40b4-8bc7-4ac1d5a79700 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:45.493003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:45.501004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ea289-312a-40b4-8bc7-4ac1d5a79700 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:45.505703 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e931401-8b00-430c-88fb-111e809271fe tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89699e3a4ec40e59d8ec0e2176c4fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616a57536e624f53a127f07dc9415713', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:45.506368 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931401-8b00-430c-88fb-111e809271fe tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:45.506652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931401-8b00-430c-88fb-111e809271fe tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:45.506930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931401-8b00-430c-88fb-111e809271fe tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:45.507662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e931401-8b00-430c-88fb-111e809271fe tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:45.507974 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e931401-8b00-430c-88fb-111e809271fe tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:45.509639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e931401-8b00-430c-88fb-111e809271fe tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:45.523346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1209/2352] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:55:45.530845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa553ebb-af33-4ebe-9de3-1f8fedef50f4 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:45.540630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa553ebb-af33-4ebe-9de3-1f8fedef50f4 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:45.545595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd1fb-4317-44d6-9365-a46fb3b1c840 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89699e3a4ec40e59d8ec0e2176c4fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616a57536e624f53a127f07dc9415713', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:45.546270 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd1fb-4317-44d6-9365-a46fb3b1c840 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:45.546562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd1fb-4317-44d6-9365-a46fb3b1c840 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:45.546854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd1fb-4317-44d6-9365-a46fb3b1c840 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:45.547643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fd1fb-4317-44d6-9365-a46fb3b1c840 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:45.549769 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fd1fb-4317-44d6-9365-a46fb3b1c840 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:45.580765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1144/2353] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:45 2022] POST /identity/v3/projects => generated 353 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:45.584706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e931401-8b00-430c-88fb-111e809271fe tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:45.595904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e931401-8b00-430c-88fb-111e809271fe tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:45.660043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244aea98-287f-405e-bee0-f55fcf2f0982 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89699e3a4ec40e59d8ec0e2176c4fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616a57536e624f53a127f07dc9415713', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:45.660696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244aea98-287f-405e-bee0-f55fcf2f0982 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:45.661051 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244aea98-287f-405e-bee0-f55fcf2f0982 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:45.661560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244aea98-287f-405e-bee0-f55fcf2f0982 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:45.662367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244aea98-287f-405e-bee0-f55fcf2f0982 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:45.662786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244aea98-287f-405e-bee0-f55fcf2f0982 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:45.664526 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244aea98-287f-405e-bee0-f55fcf2f0982 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:45.678817 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1210/2354] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:55:45.722438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd1fb-4317-44d6-9365-a46fb3b1c840 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:45.733404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd1fb-4317-44d6-9365-a46fb3b1c840 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:45.739315 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c89699e3a4ec40e59d8ec0e2176c4fa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616a57536e624f53a127f07dc9415713', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:45.740284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:45.740573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:45.740856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] PATH_INFO: `/v3/projects/fbde81ea968a4b3da2af2845fd7d268a` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:45.742118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fbde81ea968a4b3da2af2845fd7d268a)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:45.744030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:45.792338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbde81ea968a4b3da2af2845fd7d268a', 'request_id': 'req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:45.802654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbde81ea968a4b3da2af2845fd7d268a', 'request_id': 'req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:45.812049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbde81ea968a4b3da2af2845fd7d268a', 'request_id': 'req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:45.831257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] The token cache is being invalidate because project fbde81ea968a4b3da2af2845fd7d268a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:45.831529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:45.832107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bb76f032-6a2c-4f03-b946-9ca0cf631fe1 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:45.833776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1145/2355] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:45 2022] DELETE /identity/v3/projects/fbde81ea968a4b3da2af2845fd7d268a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:46.039155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244aea98-287f-405e-bee0-f55fcf2f0982 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:46.040964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-244aea98-287f-405e-bee0-f55fcf2f0982 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:46.106949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244aea98-287f-405e-bee0-f55fcf2f0982 tempest-VolumeQuotaClassesTest-761704093 tempest-VolumeQuotaClassesTest-761704093-project-admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:46.171411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:46.172724 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:46.173170 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:46.173643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] PATH_INFO: `/v3/users/4eec9efdcafc4afebf8bd047d8d205f3` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:46.175380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eec9efdcafc4afebf8bd047d8d205f3)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:46.178436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:46.231323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:46.250480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eec9efdcafc4afebf8bd047d8d205f3'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:46.267143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eec9efdcafc4afebf8bd047d8d205f3'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:46.276438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eec9efdcafc4afebf8bd047d8d205f3'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:46.277344 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eec9efdcafc4afebf8bd047d8d205f3'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:46.287098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eec9efdcafc4afebf8bd047d8d205f3'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:46.294715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1211/2356] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:46 2022] DELETE /identity/v3/users/4eec9efdcafc4afebf8bd047d8d205f3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:46.300898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e2d02-197d-4660-8373-e2773c7e3d59 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:46.301443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e2d02-197d-4660-8373-e2773c7e3d59 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:46.301734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e2d02-197d-4660-8373-e2773c7e3d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:46.324141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ce6e2d02-197d-4660-8373-e2773c7e3d59 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:46.325711 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ce6e2d02-197d-4660-8373-e2773c7e3d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:46.402334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:46.427581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1146/2357] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:46.438606 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:46.440254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:46.500285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f82a2-f150-4388-9964-f7f2d858d28f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:46.504053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700b491-ea12-4c35-a28e-c67eed165229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:46.504721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700b491-ea12-4c35-a28e-c67eed165229 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:46.505007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700b491-ea12-4c35-a28e-c67eed165229 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:46.505291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700b491-ea12-4c35-a28e-c67eed165229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:46.514971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700b491-ea12-4c35-a28e-c67eed165229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:46.516419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700b491-ea12-4c35-a28e-c67eed165229 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:46.525935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1212/2358] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:46.833698 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e2d02-197d-4660-8373-e2773c7e3d59 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:46.835149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ce6e2d02-197d-4660-8373-e2773c7e3d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:46.847825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e2d02-197d-4660-8373-e2773c7e3d59 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:46.852897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:46.853821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:46.854106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:46.854382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] PATH_INFO: `/v3/projects/ae27e18c8a5c406eab5501467e563444` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:46.855631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae27e18c8a5c406eab5501467e563444)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:46.857686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:46.913240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae27e18c8a5c406eab5501467e563444', 'request_id': 'req-8b34ede4-cf48-4751-b925-3667156d9d14'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:46.921394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae27e18c8a5c406eab5501467e563444', 'request_id': 'req-8b34ede4-cf48-4751-b925-3667156d9d14'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:46.927765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae27e18c8a5c406eab5501467e563444', 'request_id': 'req-8b34ede4-cf48-4751-b925-3667156d9d14'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:46.947124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] The token cache is being invalidate because project ae27e18c8a5c406eab5501467e563444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:46.947404 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:46.948048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:46.949656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1147/2359] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:46 2022] DELETE /identity/v3/projects/ae27e18c8a5c406eab5501467e563444 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:46.954185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700b491-ea12-4c35-a28e-c67eed165229 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:46.964623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700b491-ea12-4c35-a28e-c67eed165229 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:47.030841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fcf04a-a914-4f2f-89aa-5ac12865b292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:47.031688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcf04a-a914-4f2f-89aa-5ac12865b292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:47.031977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcf04a-a914-4f2f-89aa-5ac12865b292 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:47.032295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcf04a-a914-4f2f-89aa-5ac12865b292 admin admin] PATH_INFO: `/v3/users/c89699e3a4ec40e59d8ec0e2176c4fa8` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:47.033630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fcf04a-a914-4f2f-89aa-5ac12865b292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c89699e3a4ec40e59d8ec0e2176c4fa8)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:47.035589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fcf04a-a914-4f2f-89aa-5ac12865b292 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:47.084348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-70fcf04a-a914-4f2f-89aa-5ac12865b292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c89699e3a4ec40e59d8ec0e2176c4fa8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:47.089566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:47.099053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-70fcf04a-a914-4f2f-89aa-5ac12865b292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c89699e3a4ec40e59d8ec0e2176c4fa8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:47.107529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-70fcf04a-a914-4f2f-89aa-5ac12865b292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c89699e3a4ec40e59d8ec0e2176c4fa8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:47.108370 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-70fcf04a-a914-4f2f-89aa-5ac12865b292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c89699e3a4ec40e59d8ec0e2176c4fa8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:47.123985 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-70fcf04a-a914-4f2f-89aa-5ac12865b292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c89699e3a4ec40e59d8ec0e2176c4fa8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:47.130961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1213/2360] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:46 2022] DELETE /identity/v3/users/c89699e3a4ec40e59d8ec0e2176c4fa8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:47.430693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:47.432265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:47.446159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34ede4-cf48-4751-b925-3667156d9d14 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:47.509265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:47.510166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:47.510446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:47.510717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] PATH_INFO: `/v3/projects/616a57536e624f53a127f07dc9415713` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:47.511897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=616a57536e624f53a127f07dc9415713)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:47.513870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:47.562074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '616a57536e624f53a127f07dc9415713', 'request_id': 'req-090ec93b-e7c7-43ce-a90f-451269c47e91'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:47.568296 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '616a57536e624f53a127f07dc9415713', 'request_id': 'req-090ec93b-e7c7-43ce-a90f-451269c47e91'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:47.574012 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '616a57536e624f53a127f07dc9415713', 'request_id': 'req-090ec93b-e7c7-43ce-a90f-451269c47e91'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:47.589030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] The token cache is being invalidate because project 616a57536e624f53a127f07dc9415713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:47.589338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:47.590029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-090ec93b-e7c7-43ce-a90f-451269c47e91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:47.591584 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1148/2361] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:47 2022] DELETE /identity/v3/projects/616a57536e624f53a127f07dc9415713 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:47.597061 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf37337-8d25-496f-8e75-0ba606ea6958 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:47.597347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf37337-8d25-496f-8e75-0ba606ea6958 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:47.597676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf37337-8d25-496f-8e75-0ba606ea6958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:47.617133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-adf37337-8d25-496f-8e75-0ba606ea6958 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:47.618397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-adf37337-8d25-496f-8e75-0ba606ea6958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:47.680021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1214/2362] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:47.689497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce90aa0-093a-43f9-a679-5dd115c10b06 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:47.689787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce90aa0-093a-43f9-a679-5dd115c10b06 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:47.690067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce90aa0-093a-43f9-a679-5dd115c10b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:47.696462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:47.710817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-6ce90aa0-093a-43f9-a679-5dd115c10b06 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:47.712072 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6ce90aa0-093a-43f9-a679-5dd115c10b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:47.720224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1149/2363] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:47.724777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf37337-8d25-496f-8e75-0ba606ea6958 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:47.735659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf37337-8d25-496f-8e75-0ba606ea6958 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:47.740846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015adc08-2f49-4c3b-97b4-e1f870f2d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:47.741551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015adc08-2f49-4c3b-97b4-e1f870f2d626 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:47.741865 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015adc08-2f49-4c3b-97b4-e1f870f2d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:47.742144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015adc08-2f49-4c3b-97b4-e1f870f2d626 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:47.742880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015adc08-2f49-4c3b-97b4-e1f870f2d626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:47.743190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015adc08-2f49-4c3b-97b4-e1f870f2d626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:47.744789 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015adc08-2f49-4c3b-97b4-e1f870f2d626 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:47.757809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1215/2364] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:55:47.762525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce90aa0-093a-43f9-a679-5dd115c10b06 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:47.772927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce90aa0-093a-43f9-a679-5dd115c10b06 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:47.777579 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30577f48-fab3-4f62-b57a-b01d33624ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:47.778229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30577f48-fab3-4f62-b57a-b01d33624ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:47.778509 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30577f48-fab3-4f62-b57a-b01d33624ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:47.778786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30577f48-fab3-4f62-b57a-b01d33624ea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:47.779470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30577f48-fab3-4f62-b57a-b01d33624ea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:47.781398 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30577f48-fab3-4f62-b57a-b01d33624ea5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:47.814667 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1150/2365] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:47 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:47.818897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015adc08-2f49-4c3b-97b4-e1f870f2d626 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:47.831192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015adc08-2f49-4c3b-97b4-e1f870f2d626 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:47.888003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77516c71-b812-4d21-b8aa-d80151064d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:47.888686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77516c71-b812-4d21-b8aa-d80151064d3e admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:47.888970 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77516c71-b812-4d21-b8aa-d80151064d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:47.889248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77516c71-b812-4d21-b8aa-d80151064d3e admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:47.895168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77516c71-b812-4d21-b8aa-d80151064d3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:47.895168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77516c71-b812-4d21-b8aa-d80151064d3e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:47.914421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1216/2366] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:47 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:47.919218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30577f48-fab3-4f62-b57a-b01d33624ea5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:47.932638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30577f48-fab3-4f62-b57a-b01d33624ea5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:47.939930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f31d3-aff6-4aa3-8c0a-5d286a4a0b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:47.940681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f31d3-aff6-4aa3-8c0a-5d286a4a0b9b admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:47.941118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f31d3-aff6-4aa3-8c0a-5d286a4a0b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:47.941547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f31d3-aff6-4aa3-8c0a-5d286a4a0b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:47.942644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f31d3-aff6-4aa3-8c0a-5d286a4a0b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:47.944718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f31d3-aff6-4aa3-8c0a-5d286a4a0b9b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:47.958132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1151/2367] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:55:47.962375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77516c71-b812-4d21-b8aa-d80151064d3e admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:47.974827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77516c71-b812-4d21-b8aa-d80151064d3e admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:47.979954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213aa59-fd53-45c0-bbba-e5ff4b58ac72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:47.980633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213aa59-fd53-45c0-bbba-e5ff4b58ac72 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:47.980916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213aa59-fd53-45c0-bbba-e5ff4b58ac72 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:47.981189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213aa59-fd53-45c0-bbba-e5ff4b58ac72 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:47.981977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213aa59-fd53-45c0-bbba-e5ff4b58ac72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:47.984096 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213aa59-fd53-45c0-bbba-e5ff4b58ac72 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:47.995509 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1217/2368] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:48.000818 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f31d3-aff6-4aa3-8c0a-5d286a4a0b9b admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:48.013009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f31d3-aff6-4aa3-8c0a-5d286a4a0b9b admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:48.018015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d73452-e73e-47fd-9c24-a595c419cd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:48.018941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d73452-e73e-47fd-9c24-a595c419cd5b admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.019227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d73452-e73e-47fd-9c24-a595c419cd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.019508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d73452-e73e-47fd-9c24-a595c419cd5b admin admin] PATH_INFO: `/v3/projects/e03b61c23f53471db4a39391b5c0c0b5/users/3ee0a4a9c3954c0b8cde7aa54973f6dd/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.038071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d73452-e73e-47fd-9c24-a595c419cd5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03b61c23f53471db4a39391b5c0c0b5, user_id=3ee0a4a9c3954c0b8cde7aa54973f6dd, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:48.040749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d73452-e73e-47fd-9c24-a595c419cd5b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:48.055929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1152/2369] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:47 2022] PUT /identity/v3/projects/e03b61c23f53471db4a39391b5c0c0b5/users/3ee0a4a9c3954c0b8cde7aa54973f6dd/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:48.066818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b42843-2cd7-4899-a327-fc5cc9e44d53 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.067007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b42843-2cd7-4899-a327-fc5cc9e44d53 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.067278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b42843-2cd7-4899-a327-fc5cc9e44d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.097382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f6b42843-2cd7-4899-a327-fc5cc9e44d53 None None] MFA Rules not processed for user `3ee0a4a9c3954c0b8cde7aa54973f6dd`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:48.098624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f6b42843-2cd7-4899-a327-fc5cc9e44d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:48.127199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:48.175318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1218/2370] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:48.179845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d73452-e73e-47fd-9c24-a595c419cd5b admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:48.243520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d73452-e73e-47fd-9c24-a595c419cd5b admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:48.248309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b0558-5c4a-4a33-bd69-f1fa566497c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:48.249017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b0558-5c4a-4a33-bd69-f1fa566497c4 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.249303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b0558-5c4a-4a33-bd69-f1fa566497c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.249613 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b0558-5c4a-4a33-bd69-f1fa566497c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.250312 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b0558-5c4a-4a33-bd69-f1fa566497c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:48.252250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b0558-5c4a-4a33-bd69-f1fa566497c4 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:48.278565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1153/2371] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:48 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:48.283312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b42843-2cd7-4899-a327-fc5cc9e44d53 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:48.294166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b42843-2cd7-4899-a327-fc5cc9e44d53 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:48.353463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b35a2a-510b-44fb-bf53-ee95e90f1e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:48.354054 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b35a2a-510b-44fb-bf53-ee95e90f1e0c admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.354337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b35a2a-510b-44fb-bf53-ee95e90f1e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.354619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b35a2a-510b-44fb-bf53-ee95e90f1e0c admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.355299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b35a2a-510b-44fb-bf53-ee95e90f1e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:48.357196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b35a2a-510b-44fb-bf53-ee95e90f1e0c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:48.381624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1219/2372] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:48 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:55:48.385564 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b0558-5c4a-4a33-bd69-f1fa566497c4 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:48.399834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b0558-5c4a-4a33-bd69-f1fa566497c4 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:48.404715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746e21a-c902-41e0-8912-4c6419d84658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:48.406069 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746e21a-c902-41e0-8912-4c6419d84658 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.406359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746e21a-c902-41e0-8912-4c6419d84658 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.406641 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746e21a-c902-41e0-8912-4c6419d84658 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.407386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746e21a-c902-41e0-8912-4c6419d84658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:48.409008 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746e21a-c902-41e0-8912-4c6419d84658 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:48.420489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1154/2373] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:48.424644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b35a2a-510b-44fb-bf53-ee95e90f1e0c admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:48.434755 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b35a2a-510b-44fb-bf53-ee95e90f1e0c admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:48.439430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aec1ed-7dd5-4e7f-a928-d8fec309ac7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:48.440083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aec1ed-7dd5-4e7f-a928-d8fec309ac7e admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.440364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aec1ed-7dd5-4e7f-a928-d8fec309ac7e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.440637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aec1ed-7dd5-4e7f-a928-d8fec309ac7e admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.441387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aec1ed-7dd5-4e7f-a928-d8fec309ac7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:48.443067 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aec1ed-7dd5-4e7f-a928-d8fec309ac7e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:48.453779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1220/2374] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:55:48.458175 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746e21a-c902-41e0-8912-4c6419d84658 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:48.469329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746e21a-c902-41e0-8912-4c6419d84658 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:48.480702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc530498-ac8c-4930-b04e-d65d0fe574e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:48.481940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530498-ac8c-4930-b04e-d65d0fe574e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.482276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530498-ac8c-4930-b04e-d65d0fe574e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.482571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc530498-ac8c-4930-b04e-d65d0fe574e7 admin admin] PATH_INFO: `/v3/projects/cc992377d98b42098d669a136208f35d/users/5252b160ca2c4847a1f03469c1b642dd/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.501093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc530498-ac8c-4930-b04e-d65d0fe574e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc992377d98b42098d669a136208f35d, user_id=5252b160ca2c4847a1f03469c1b642dd, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:48.503821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc530498-ac8c-4930-b04e-d65d0fe574e7 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:48.517567 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1155/2375] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:48 2022] PUT /identity/v3/projects/cc992377d98b42098d669a136208f35d/users/5252b160ca2c4847a1f03469c1b642dd/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:48.528259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700ea69-24e3-4a2c-8be8-ae333e4a31b2 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.528544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700ea69-24e3-4a2c-8be8-ae333e4a31b2 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.528852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700ea69-24e3-4a2c-8be8-ae333e4a31b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.557641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-6700ea69-24e3-4a2c-8be8-ae333e4a31b2 None None] MFA Rules not processed for user `5252b160ca2c4847a1f03469c1b642dd`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:48.558888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6700ea69-24e3-4a2c-8be8-ae333e4a31b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:48.638299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1221/2376] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:48.646076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc530498-ac8c-4930-b04e-d65d0fe574e7 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:48.707396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:48.725795 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc530498-ac8c-4930-b04e-d65d0fe574e7 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:48.731119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39e528-f66b-4b3d-847d-cf24b2389ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:48.731803 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39e528-f66b-4b3d-847d-cf24b2389ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.732088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39e528-f66b-4b3d-847d-cf24b2389ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.732372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39e528-f66b-4b3d-847d-cf24b2389ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.733079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39e528-f66b-4b3d-847d-cf24b2389ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:48.735105 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39e528-f66b-4b3d-847d-cf24b2389ca5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:48.761195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1156/2377] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:48 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:48.765853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700ea69-24e3-4a2c-8be8-ae333e4a31b2 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:48.777329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700ea69-24e3-4a2c-8be8-ae333e4a31b2 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:48.814185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:48.842239 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8280ff7-4d87-4751-af00-650a8154ad0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:48.842973 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8280ff7-4d87-4751-af00-650a8154ad0c admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.843253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8280ff7-4d87-4751-af00-650a8154ad0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.843527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8280ff7-4d87-4751-af00-650a8154ad0c admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.844206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8280ff7-4d87-4751-af00-650a8154ad0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:48.846148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8280ff7-4d87-4751-af00-650a8154ad0c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:48.874460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1222/2378] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:48 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:48.880836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39e528-f66b-4b3d-847d-cf24b2389ca5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:48.892533 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39e528-f66b-4b3d-847d-cf24b2389ca5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:48.897271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671866c2-86ac-45e1-b3a4-f4803189b0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:48.898015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671866c2-86ac-45e1-b3a4-f4803189b0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.898302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671866c2-86ac-45e1-b3a4-f4803189b0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.898581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671866c2-86ac-45e1-b3a4-f4803189b0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.899321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671866c2-86ac-45e1-b3a4-f4803189b0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:48.900924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671866c2-86ac-45e1-b3a4-f4803189b0f8 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:48.912104 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1157/2379] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:55:48.916328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8280ff7-4d87-4751-af00-650a8154ad0c admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:48.926988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8280ff7-4d87-4751-af00-650a8154ad0c admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:48.931686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f533a-9da4-49f3-8c5a-66456f41ac5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:48.932592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f533a-9da4-49f3-8c5a-66456f41ac5d admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:48.932875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f533a-9da4-49f3-8c5a-66456f41ac5d admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:48.933148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f533a-9da4-49f3-8c5a-66456f41ac5d admin admin] PATH_INFO: `/v3/projects/d50e99fe4ef2470ebdd627c50d565632/users/81c255b4cf524b1795c47cbf41f4ae65/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:48.949145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f533a-9da4-49f3-8c5a-66456f41ac5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50e99fe4ef2470ebdd627c50d565632, user_id=81c255b4cf524b1795c47cbf41f4ae65, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:48.951838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f533a-9da4-49f3-8c5a-66456f41ac5d admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:48.973462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1223/2380] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:48 2022] PUT /identity/v3/projects/d50e99fe4ef2470ebdd627c50d565632/users/81c255b4cf524b1795c47cbf41f4ae65/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:48.976586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671866c2-86ac-45e1-b3a4-f4803189b0f8 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:48.987475 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671866c2-86ac-45e1-b3a4-f4803189b0f8 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:49.046925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29190df4-fbeb-4a9d-a836-fc6375932ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:49.047635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29190df4-fbeb-4a9d-a836-fc6375932ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:49.047920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29190df4-fbeb-4a9d-a836-fc6375932ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:49.048198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29190df4-fbeb-4a9d-a836-fc6375932ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:49.048971 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29190df4-fbeb-4a9d-a836-fc6375932ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:49.050666 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29190df4-fbeb-4a9d-a836-fc6375932ef9 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:49.062667 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1158/2381] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:49.067247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f533a-9da4-49f3-8c5a-66456f41ac5d admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:49.078563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f533a-9da4-49f3-8c5a-66456f41ac5d admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:49.083696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31324b10-35dc-4d50-8fd4-d83f658ec2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:49.084627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31324b10-35dc-4d50-8fd4-d83f658ec2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:49.084908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31324b10-35dc-4d50-8fd4-d83f658ec2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:49.085186 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31324b10-35dc-4d50-8fd4-d83f658ec2fd admin admin] PATH_INFO: `/v3/projects/d50e99fe4ef2470ebdd627c50d565632/users/81c255b4cf524b1795c47cbf41f4ae65/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:49.087328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31324b10-35dc-4d50-8fd4-d83f658ec2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50e99fe4ef2470ebdd627c50d565632, user_id=81c255b4cf524b1795c47cbf41f4ae65, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:49.090397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31324b10-35dc-4d50-8fd4-d83f658ec2fd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:49.101788 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1224/2382] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:49 2022] PUT /identity/v3/projects/d50e99fe4ef2470ebdd627c50d565632/users/81c255b4cf524b1795c47cbf41f4ae65/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:49.113837 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8179-dc5c-4d53-b174-abcbf06ea04d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:49.114230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8179-dc5c-4d53-b174-abcbf06ea04d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:49.114643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8179-dc5c-4d53-b174-abcbf06ea04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:49.145532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-274d8179-dc5c-4d53-b174-abcbf06ea04d None None] MFA Rules not processed for user `81c255b4cf524b1795c47cbf41f4ae65`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:49.146680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-274d8179-dc5c-4d53-b174-abcbf06ea04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:49.223260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1159/2383] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:49.235410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31324b10-35dc-4d50-8fd4-d83f658ec2fd admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:49.237212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-31324b10-35dc-4d50-8fd4-d83f658ec2fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:49.315846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31324b10-35dc-4d50-8fd4-d83f658ec2fd admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:49.319648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582103a-6984-4c40-adff-6da8a9847f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:49.320321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582103a-6984-4c40-adff-6da8a9847f6e service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:49.320664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582103a-6984-4c40-adff-6da8a9847f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:49.320945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582103a-6984-4c40-adff-6da8a9847f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:49.330966 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582103a-6984-4c40-adff-6da8a9847f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:49.332397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582103a-6984-4c40-adff-6da8a9847f6e service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:49.342347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1225/2384] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:49.894835 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d8179-dc5c-4d53-b174-abcbf06ea04d None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:49.896482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-274d8179-dc5c-4d53-b174-abcbf06ea04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:49.910056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d8179-dc5c-4d53-b174-abcbf06ea04d None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:49.913525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c50313-4367-4580-8c0a-d4ea906ca9d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:49.914194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50313-4367-4580-8c0a-d4ea906ca9d9 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:49.914471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50313-4367-4580-8c0a-d4ea906ca9d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:49.914748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50313-4367-4580-8c0a-d4ea906ca9d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:49.960091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:49.988184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c50313-4367-4580-8c0a-d4ea906ca9d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:49.989904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c50313-4367-4580-8c0a-d4ea906ca9d9 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:50.001791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1160/2385] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:54.031190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582103a-6984-4c40-adff-6da8a9847f6e service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:54.040410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582103a-6984-4c40-adff-6da8a9847f6e service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:54.044345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751946a-378d-409b-a920-a8eddb95723a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:54.045000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751946a-378d-409b-a920-a8eddb95723a service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:54.045286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751946a-378d-409b-a920-a8eddb95723a service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:54.045770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751946a-378d-409b-a920-a8eddb95723a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:54.105662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751946a-378d-409b-a920-a8eddb95723a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:54.107014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751946a-378d-409b-a920-a8eddb95723a service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:54.115914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1226/2386] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:56.364911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c50313-4367-4580-8c0a-d4ea906ca9d9 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:56.429514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c50313-4367-4580-8c0a-d4ea906ca9d9 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:56.433998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:56.434858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:56.435145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:56.435416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] PATH_INFO: `/v3/users/3ee0a4a9c3954c0b8cde7aa54973f6dd` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:56.436594 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee0a4a9c3954c0b8cde7aa54973f6dd)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:56.439080 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:56.486964 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee0a4a9c3954c0b8cde7aa54973f6dd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:56.495200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee0a4a9c3954c0b8cde7aa54973f6dd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:56.502534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee0a4a9c3954c0b8cde7aa54973f6dd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:56.514259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee0a4a9c3954c0b8cde7aa54973f6dd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:56.514944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee0a4a9c3954c0b8cde7aa54973f6dd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:56.521247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1161/2387] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:56 2022] DELETE /identity/v3/users/3ee0a4a9c3954c0b8cde7aa54973f6dd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:56.526209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecd801-a85f-4231-b73f-3566d9245401 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:56.526693 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecd801-a85f-4231-b73f-3566d9245401 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:56.526787 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecd801-a85f-4231-b73f-3566d9245401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:56.544819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-35ecd801-a85f-4231-b73f-3566d9245401 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:56.546106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-35ecd801-a85f-4231-b73f-3566d9245401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:56.571030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:56.609934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1227/2388] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:56.619533 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:56.620980 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:56.683393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6120b-3a14-4588-b65c-aefe76862820 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:56.687608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b90b81-f8ec-4830-973d-3cf6d7880d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:56.688261 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b90b81-f8ec-4830-973d-3cf6d7880d0c service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:56.688547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b90b81-f8ec-4830-973d-3cf6d7880d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:56.688823 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b90b81-f8ec-4830-973d-3cf6d7880d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:56.698630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b90b81-f8ec-4830-973d-3cf6d7880d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:56.700055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b90b81-f8ec-4830-973d-3cf6d7880d0c service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:56.710177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1162/2389] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:57.004309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecd801-a85f-4231-b73f-3566d9245401 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:57.005901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-35ecd801-a85f-4231-b73f-3566d9245401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:57.019777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecd801-a85f-4231-b73f-3566d9245401 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:57.025154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:57.026535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:57.026995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:57.027408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] PATH_INFO: `/v3/projects/e03b61c23f53471db4a39391b5c0c0b5` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:57.030153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03b61c23f53471db4a39391b5c0c0b5)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:57.033092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:57.082650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03b61c23f53471db4a39391b5c0c0b5', 'request_id': 'req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.089912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03b61c23f53471db4a39391b5c0c0b5', 'request_id': 'req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.095659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03b61c23f53471db4a39391b5c0c0b5', 'request_id': 'req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.107324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] The token cache is being invalidate because project e03b61c23f53471db4a39391b5c0c0b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:57.107527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.108162 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.110303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1228/2390] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:57 2022] DELETE /identity/v3/projects/e03b61c23f53471db4a39391b5c0c0b5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:57.114575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b90b81-f8ec-4830-973d-3cf6d7880d0c service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:57.120716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:57.125199 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b90b81-f8ec-4830-973d-3cf6d7880d0c service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:57.187604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:57.188210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:57.188493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:57.188764 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] PATH_INFO: `/v3/users/5252b160ca2c4847a1f03469c1b642dd` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:57.189985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5252b160ca2c4847a1f03469c1b642dd)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:57.191886 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:57.239936 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5252b160ca2c4847a1f03469c1b642dd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.247888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5252b160ca2c4847a1f03469c1b642dd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.255397 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5252b160ca2c4847a1f03469c1b642dd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.267034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5252b160ca2c4847a1f03469c1b642dd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.267681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5252b160ca2c4847a1f03469c1b642dd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.273486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1163/2391] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:57 2022] DELETE /identity/v3/users/5252b160ca2c4847a1f03469c1b642dd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:57.563618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:57.565460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:57.580794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24d99c-c69e-4a1d-9d0f-48b3ee83be30 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:57.642743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:57.643519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:57.643803 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:57.644077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] PATH_INFO: `/v3/projects/cc992377d98b42098d669a136208f35d` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:57.645258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc992377d98b42098d669a136208f35d)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:57.647186 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:57.694114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc992377d98b42098d669a136208f35d', 'request_id': 'req-a849ac93-9e35-4d8e-9220-c266483a9e69'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.697699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:57.701717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc992377d98b42098d669a136208f35d', 'request_id': 'req-a849ac93-9e35-4d8e-9220-c266483a9e69'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.709772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc992377d98b42098d669a136208f35d', 'request_id': 'req-a849ac93-9e35-4d8e-9220-c266483a9e69'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.721317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] The token cache is being invalidate because project cc992377d98b42098d669a136208f35d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:57.721627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.722284 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.723820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1229/2392] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:57 2022] DELETE /identity/v3/projects/cc992377d98b42098d669a136208f35d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:57.728135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:57.729767 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:57.743074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad2e2a-d659-4505-afc7-ac860a88fc10 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:57.803602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1326c-3b65-448c-95d1-422d6a8b27d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:57.804495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1326c-3b65-448c-95d1-422d6a8b27d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:57.804800 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1326c-3b65-448c-95d1-422d6a8b27d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:57.805060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1326c-3b65-448c-95d1-422d6a8b27d2 admin admin] PATH_INFO: `/v3/users/81c255b4cf524b1795c47cbf41f4ae65` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:57.806321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1326c-3b65-448c-95d1-422d6a8b27d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c255b4cf524b1795c47cbf41f4ae65)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:57.808249 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1326c-3b65-448c-95d1-422d6a8b27d2 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:57.839566 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:57.861348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2bb1326c-3b65-448c-95d1-422d6a8b27d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c255b4cf524b1795c47cbf41f4ae65'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.869082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2bb1326c-3b65-448c-95d1-422d6a8b27d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c255b4cf524b1795c47cbf41f4ae65'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.877641 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2bb1326c-3b65-448c-95d1-422d6a8b27d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c255b4cf524b1795c47cbf41f4ae65'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.892229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2bb1326c-3b65-448c-95d1-422d6a8b27d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c255b4cf524b1795c47cbf41f4ae65'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.893014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-2bb1326c-3b65-448c-95d1-422d6a8b27d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c255b4cf524b1795c47cbf41f4ae65'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:57.899091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1164/2393] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:57 2022] DELETE /identity/v3/users/81c255b4cf524b1795c47cbf41f4ae65 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:58.189564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:58.191095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:58.208342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849ac93-9e35-4d8e-9220-c266483a9e69 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.275138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.276009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.276404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.276707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] PATH_INFO: `/v3/projects/d50e99fe4ef2470ebdd627c50d565632` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.278107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d50e99fe4ef2470ebdd627c50d565632)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.280181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:58.328760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd50e99fe4ef2470ebdd627c50d565632', 'request_id': 'req-139fb6cf-daec-43b7-a1a6-3e4307c56cff'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.336251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd50e99fe4ef2470ebdd627c50d565632', 'request_id': 'req-139fb6cf-daec-43b7-a1a6-3e4307c56cff'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.342023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd50e99fe4ef2470ebdd627c50d565632', 'request_id': 'req-139fb6cf-daec-43b7-a1a6-3e4307c56cff'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.353294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] The token cache is being invalidate because project d50e99fe4ef2470ebdd627c50d565632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:58.353610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.354262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-139fb6cf-daec-43b7-a1a6-3e4307c56cff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.355787 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1230/2394] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:58 2022] DELETE /identity/v3/projects/d50e99fe4ef2470ebdd627c50d565632 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:58.362201 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64a0e2-4166-4a4b-b11c-c6b5eaa8ae96 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.362500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64a0e2-4166-4a4b-b11c-c6b5eaa8ae96 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.362789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64a0e2-4166-4a4b-b11c-c6b5eaa8ae96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.382303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-cb64a0e2-4166-4a4b-b11c-c6b5eaa8ae96 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:58.383532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-cb64a0e2-4166-4a4b-b11c-c6b5eaa8ae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:58.442371 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1165/2395] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:58.450696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f98248-66a1-481d-99ab-3720a55d9c99 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.451049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f98248-66a1-481d-99ab-3720a55d9c99 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.451338 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f98248-66a1-481d-99ab-3720a55d9c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.471076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-c7f98248-66a1-481d-99ab-3720a55d9c99 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:58.472313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c7f98248-66a1-481d-99ab-3720a55d9c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:58.482014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1231/2396] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:58.487046 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64a0e2-4166-4a4b-b11c-c6b5eaa8ae96 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:58.496752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64a0e2-4166-4a4b-b11c-c6b5eaa8ae96 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.502216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41357d66-ecc2-4d45-a1f2-058f251a19f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.502912 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357d66-ecc2-4d45-a1f2-058f251a19f0 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.503468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357d66-ecc2-4d45-a1f2-058f251a19f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.503900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357d66-ecc2-4d45-a1f2-058f251a19f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.505233 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41357d66-ecc2-4d45-a1f2-058f251a19f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:58.505517 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41357d66-ecc2-4d45-a1f2-058f251a19f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.508257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41357d66-ecc2-4d45-a1f2-058f251a19f0 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:58.522868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1166/2397] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:55:58.527237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f98248-66a1-481d-99ab-3720a55d9c99 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:58.536548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f98248-66a1-481d-99ab-3720a55d9c99 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.541772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97fd00-2217-4139-9052-d02cfd8ed214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.542425 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97fd00-2217-4139-9052-d02cfd8ed214 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.542708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97fd00-2217-4139-9052-d02cfd8ed214 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.543019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97fd00-2217-4139-9052-d02cfd8ed214 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.543704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97fd00-2217-4139-9052-d02cfd8ed214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.545611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97fd00-2217-4139-9052-d02cfd8ed214 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:58.575145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1232/2398] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:58 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:58.576297 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41357d66-ecc2-4d45-a1f2-058f251a19f0 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:58.586593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41357d66-ecc2-4d45-a1f2-058f251a19f0 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.647388 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15490e1a-ad53-4790-b204-98d7801511e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.648550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15490e1a-ad53-4790-b204-98d7801511e3 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.649080 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15490e1a-ad53-4790-b204-98d7801511e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.649626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15490e1a-ad53-4790-b204-98d7801511e3 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.650401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15490e1a-ad53-4790-b204-98d7801511e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.652344 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15490e1a-ad53-4790-b204-98d7801511e3 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:58.676206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1167/2399] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:58 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:55:58.680318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97fd00-2217-4139-9052-d02cfd8ed214 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:58.691999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97fd00-2217-4139-9052-d02cfd8ed214 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.696829 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699eefd-2b8b-40e7-99c0-fd47f7c3d91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.697460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699eefd-2b8b-40e7-99c0-fd47f7c3d91d admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.697744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699eefd-2b8b-40e7-99c0-fd47f7c3d91d admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.698018 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699eefd-2b8b-40e7-99c0-fd47f7c3d91d admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.698746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699eefd-2b8b-40e7-99c0-fd47f7c3d91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.700338 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699eefd-2b8b-40e7-99c0-fd47f7c3d91d admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:58.711472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1233/2400] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:58.716412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15490e1a-ad53-4790-b204-98d7801511e3 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:58.726867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15490e1a-ad53-4790-b204-98d7801511e3 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.732006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e9299-3b6c-4022-8a4b-d75aa6aba069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.732714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e9299-3b6c-4022-8a4b-d75aa6aba069 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.732967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e9299-3b6c-4022-8a4b-d75aa6aba069 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.733251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e9299-3b6c-4022-8a4b-d75aa6aba069 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.736252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e9299-3b6c-4022-8a4b-d75aa6aba069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.736252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e9299-3b6c-4022-8a4b-d75aa6aba069 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:58.750417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1168/2401] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:58.754808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699eefd-2b8b-40e7-99c0-fd47f7c3d91d admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:58.765173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699eefd-2b8b-40e7-99c0-fd47f7c3d91d admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.771354 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecf4a2-0e15-4dcd-b9cc-0082c397de11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.772260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecf4a2-0e15-4dcd-b9cc-0082c397de11 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.772548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecf4a2-0e15-4dcd-b9cc-0082c397de11 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.772820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecf4a2-0e15-4dcd-b9cc-0082c397de11 admin admin] PATH_INFO: `/v3/projects/b9dc40d51a074c4e8ddcd91f97d5f09c/users/8e144457daa24cad9cc70a5e318e6039/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.789132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecf4a2-0e15-4dcd-b9cc-0082c397de11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9dc40d51a074c4e8ddcd91f97d5f09c, user_id=8e144457daa24cad9cc70a5e318e6039, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.791839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecf4a2-0e15-4dcd-b9cc-0082c397de11 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:58.805857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1234/2402] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:58 2022] PUT /identity/v3/projects/b9dc40d51a074c4e8ddcd91f97d5f09c/users/8e144457daa24cad9cc70a5e318e6039/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:58.818607 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff6c2-ac00-464f-ada0-b0a30d4c8fda None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.818898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff6c2-ac00-464f-ada0-b0a30d4c8fda None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.819176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff6c2-ac00-464f-ada0-b0a30d4c8fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.836937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:58.847857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-184ff6c2-ac00-464f-ada0-b0a30d4c8fda None None] MFA Rules not processed for user `8e144457daa24cad9cc70a5e318e6039`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:58.849533 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-184ff6c2-ac00-464f-ada0-b0a30d4c8fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:58.914078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1169/2403] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:58.918587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecf4a2-0e15-4dcd-b9cc-0082c397de11 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:58.930262 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:58.986808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecf4a2-0e15-4dcd-b9cc-0082c397de11 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.991468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d81b2-49c2-43c4-9d7e-a405f88b126b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.992129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d81b2-49c2-43c4-9d7e-a405f88b126b admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.992412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d81b2-49c2-43c4-9d7e-a405f88b126b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.992687 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d81b2-49c2-43c4-9d7e-a405f88b126b admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.993392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d81b2-49c2-43c4-9d7e-a405f88b126b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.995345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d81b2-49c2-43c4-9d7e-a405f88b126b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.021815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1235/2404] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:58 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:59.026762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ff6c2-ac00-464f-ada0-b0a30d4c8fda None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:59.040075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ff6c2-ac00-464f-ada0-b0a30d4c8fda None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:59.103682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2727aa-a56f-479e-b5b7-5428ac5e5c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:59.104376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2727aa-a56f-479e-b5b7-5428ac5e5c21 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.104663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2727aa-a56f-479e-b5b7-5428ac5e5c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.104946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2727aa-a56f-479e-b5b7-5428ac5e5c21 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.105714 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2727aa-a56f-479e-b5b7-5428ac5e5c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:59.107654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2727aa-a56f-479e-b5b7-5428ac5e5c21 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.129671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1170/2405] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:59 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:59.141571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d81b2-49c2-43c4-9d7e-a405f88b126b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:59.153126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d81b2-49c2-43c4-9d7e-a405f88b126b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:59.165098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc546223-1cbb-4f2a-a744-a577b8a406b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:59.166008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc546223-1cbb-4f2a-a744-a577b8a406b9 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.166332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc546223-1cbb-4f2a-a744-a577b8a406b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.166620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc546223-1cbb-4f2a-a744-a577b8a406b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.167561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc546223-1cbb-4f2a-a744-a577b8a406b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:59.169049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc546223-1cbb-4f2a-a744-a577b8a406b9 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.182022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1236/2406] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:59.186047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2727aa-a56f-479e-b5b7-5428ac5e5c21 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:59.197802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2727aa-a56f-479e-b5b7-5428ac5e5c21 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:59.202961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1ce62-ac3f-4832-b26f-d77a7fba2b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:59.203858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1ce62-ac3f-4832-b26f-d77a7fba2b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.204145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1ce62-ac3f-4832-b26f-d77a7fba2b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.204455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1ce62-ac3f-4832-b26f-d77a7fba2b0b admin admin] PATH_INFO: `/v3/projects/0d5f69bf23c249e3b891ef39bcb1bf15/users/114d56984f8e4e14b402c96660c23622/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.221210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1ce62-ac3f-4832-b26f-d77a7fba2b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5f69bf23c249e3b891ef39bcb1bf15, user_id=114d56984f8e4e14b402c96660c23622, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:59.223906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1ce62-ac3f-4832-b26f-d77a7fba2b0b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.239163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1171/2407] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:59 2022] PUT /identity/v3/projects/0d5f69bf23c249e3b891ef39bcb1bf15/users/114d56984f8e4e14b402c96660c23622/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:59.243610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc546223-1cbb-4f2a-a744-a577b8a406b9 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:59.254285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc546223-1cbb-4f2a-a744-a577b8a406b9 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:59.324405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71af87-0add-4ffa-a38a-5dda2d3be260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:59.325086 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71af87-0add-4ffa-a38a-5dda2d3be260 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.325411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71af87-0add-4ffa-a38a-5dda2d3be260 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.325695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71af87-0add-4ffa-a38a-5dda2d3be260 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.326426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71af87-0add-4ffa-a38a-5dda2d3be260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:59.328545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71af87-0add-4ffa-a38a-5dda2d3be260 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.339448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1237/2408] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:59.344457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1ce62-ac3f-4832-b26f-d77a7fba2b0b admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:59.355347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1ce62-ac3f-4832-b26f-d77a7fba2b0b admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:59.360526 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbad2b-d617-41e5-b4f7-722f07dc0287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:59.361747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbad2b-d617-41e5-b4f7-722f07dc0287 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.362204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbad2b-d617-41e5-b4f7-722f07dc0287 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.362625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbad2b-d617-41e5-b4f7-722f07dc0287 admin admin] PATH_INFO: `/v3/projects/0d5f69bf23c249e3b891ef39bcb1bf15/users/114d56984f8e4e14b402c96660c23622/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.364606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbad2b-d617-41e5-b4f7-722f07dc0287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5f69bf23c249e3b891ef39bcb1bf15, user_id=114d56984f8e4e14b402c96660c23622, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:59.367335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbad2b-d617-41e5-b4f7-722f07dc0287 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.380616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1172/2409] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:59 2022] PUT /identity/v3/projects/0d5f69bf23c249e3b891ef39bcb1bf15/users/114d56984f8e4e14b402c96660c23622/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:59.394678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454308e5-bfab-4c1e-8448-758347253d75 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.394961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454308e5-bfab-4c1e-8448-758347253d75 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.395245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454308e5-bfab-4c1e-8448-758347253d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.423038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-454308e5-bfab-4c1e-8448-758347253d75 None None] MFA Rules not processed for user `114d56984f8e4e14b402c96660c23622`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:59.424245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-454308e5-bfab-4c1e-8448-758347253d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:59.495617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1238/2410] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:55:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:59.506532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbad2b-d617-41e5-b4f7-722f07dc0287 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:59.508244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ffdbad2b-d617-41e5-b4f7-722f07dc0287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:59.568781 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbad2b-d617-41e5-b4f7-722f07dc0287 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:59.572354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3cfe4-cef8-42ff-ae65-8d469c986e8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:59.572996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3cfe4-cef8-42ff-ae65-8d469c986e8e service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.573278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3cfe4-cef8-42ff-ae65-8d469c986e8e service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.573580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3cfe4-cef8-42ff-ae65-8d469c986e8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.580031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:59.583591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3cfe4-cef8-42ff-ae65-8d469c986e8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:59.585630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3cfe4-cef8-42ff-ae65-8d469c986e8e service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.596331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1173/2411] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:00.200483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454308e5-bfab-4c1e-8448-758347253d75 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.205957 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:00.265277 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454308e5-bfab-4c1e-8448-758347253d75 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.269954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.270839 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.271124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.271404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] PATH_INFO: `/v3/users/8e144457daa24cad9cc70a5e318e6039` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.272591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e144457daa24cad9cc70a5e318e6039)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.274544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.320928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e144457daa24cad9cc70a5e318e6039'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.332786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e144457daa24cad9cc70a5e318e6039'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.339630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e144457daa24cad9cc70a5e318e6039'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.340303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e144457daa24cad9cc70a5e318e6039'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.347450 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e144457daa24cad9cc70a5e318e6039'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.353574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1239/2412] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:00 2022] DELETE /identity/v3/users/8e144457daa24cad9cc70a5e318e6039 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:00.358105 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37508fe-550d-48b0-add8-508bf2302540 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.358591 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37508fe-550d-48b0-add8-508bf2302540 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.358715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37508fe-550d-48b0-add8-508bf2302540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.377331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-d37508fe-550d-48b0-add8-508bf2302540 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:00.378584 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d37508fe-550d-48b0-add8-508bf2302540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.439150 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1174/2413] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:00.449069 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.450544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.509180 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06ce83-1961-48b4-9602-557db65da557 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.513011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bcca6-7a37-42ec-91ad-8a1373261816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.513702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bcca6-7a37-42ec-91ad-8a1373261816 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.513991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bcca6-7a37-42ec-91ad-8a1373261816 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.514297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bcca6-7a37-42ec-91ad-8a1373261816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.523441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511bcca6-7a37-42ec-91ad-8a1373261816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.524804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511bcca6-7a37-42ec-91ad-8a1373261816 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.535905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1240/2414] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:00.820401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37508fe-550d-48b0-add8-508bf2302540 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.821893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d37508fe-550d-48b0-add8-508bf2302540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.834628 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37508fe-550d-48b0-add8-508bf2302540 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.839997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.840881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.841182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.841498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] PATH_INFO: `/v3/projects/b9dc40d51a074c4e8ddcd91f97d5f09c` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.842753 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9dc40d51a074c4e8ddcd91f97d5f09c)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.845294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.896993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9dc40d51a074c4e8ddcd91f97d5f09c', 'request_id': 'req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.904196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9dc40d51a074c4e8ddcd91f97d5f09c', 'request_id': 'req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.911063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9dc40d51a074c4e8ddcd91f97d5f09c', 'request_id': 'req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.924393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] The token cache is being invalidate because project b9dc40d51a074c4e8ddcd91f97d5f09c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:00.924682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.925313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.926985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1175/2415] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:00 2022] DELETE /identity/v3/projects/b9dc40d51a074c4e8ddcd91f97d5f09c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:00.931155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bcca6-7a37-42ec-91ad-8a1373261816 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.941645 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bcca6-7a37-42ec-91ad-8a1373261816 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.967383 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:01.015278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068ece6-e061-4375-996c-ad77fd2472a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.016681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068ece6-e061-4375-996c-ad77fd2472a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.016969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068ece6-e061-4375-996c-ad77fd2472a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.017252 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068ece6-e061-4375-996c-ad77fd2472a6 admin admin] PATH_INFO: `/v3/users/114d56984f8e4e14b402c96660c23622` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.018531 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068ece6-e061-4375-996c-ad77fd2472a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=114d56984f8e4e14b402c96660c23622)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.020469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068ece6-e061-4375-996c-ad77fd2472a6 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.075363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-9068ece6-e061-4375-996c-ad77fd2472a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114d56984f8e4e14b402c96660c23622'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.088042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-9068ece6-e061-4375-996c-ad77fd2472a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114d56984f8e4e14b402c96660c23622'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.096675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-9068ece6-e061-4375-996c-ad77fd2472a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114d56984f8e4e14b402c96660c23622'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.097285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-9068ece6-e061-4375-996c-ad77fd2472a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114d56984f8e4e14b402c96660c23622'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.105427 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-9068ece6-e061-4375-996c-ad77fd2472a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114d56984f8e4e14b402c96660c23622'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.111669 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1241/2416] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:00 2022] DELETE /identity/v3/users/114d56984f8e4e14b402c96660c23622 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:01.392947 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.394486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.408709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288dd19-ccfe-4ff9-ad95-30e5dde5001f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.468493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.469985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.470266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.470536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] PATH_INFO: `/v3/projects/0d5f69bf23c249e3b891ef39bcb1bf15` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.471776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5f69bf23c249e3b891ef39bcb1bf15)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.473725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.497334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:01.529178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5f69bf23c249e3b891ef39bcb1bf15', 'request_id': 'req-57d5a289-53cc-4d70-b689-56d0f0eb1e50'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.536384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5f69bf23c249e3b891ef39bcb1bf15', 'request_id': 'req-57d5a289-53cc-4d70-b689-56d0f0eb1e50'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.543450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5f69bf23c249e3b891ef39bcb1bf15', 'request_id': 'req-57d5a289-53cc-4d70-b689-56d0f0eb1e50'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.555895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] The token cache is being invalidate because project 0d5f69bf23c249e3b891ef39bcb1bf15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:01.556185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.556811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-57d5a289-53cc-4d70-b689-56d0f0eb1e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.558400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1176/2417] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:01 2022] DELETE /identity/v3/projects/0d5f69bf23c249e3b891ef39bcb1bf15 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:01.563367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f8ac0-2087-4145-8a80-7c0af75141e1 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.563843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f8ac0-2087-4145-8a80-7c0af75141e1 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.563956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f8ac0-2087-4145-8a80-7c0af75141e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.582424 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-fc0f8ac0-2087-4145-8a80-7c0af75141e1 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:01.583628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fc0f8ac0-2087-4145-8a80-7c0af75141e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.642074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1242/2418] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:01.650084 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185218b7-a8e6-4e4d-9efe-66fb05951dc5 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.650412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185218b7-a8e6-4e4d-9efe-66fb05951dc5 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.650696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185218b7-a8e6-4e4d-9efe-66fb05951dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.669380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-185218b7-a8e6-4e4d-9efe-66fb05951dc5 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:01.670606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-185218b7-a8e6-4e4d-9efe-66fb05951dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.678813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1177/2419] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:01.683004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f8ac0-2087-4145-8a80-7c0af75141e1 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.692707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f8ac0-2087-4145-8a80-7c0af75141e1 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.697264 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c65f9d-60cb-4b33-aab8-154c52818cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.697954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c65f9d-60cb-4b33-aab8-154c52818cbe admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.698246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c65f9d-60cb-4b33-aab8-154c52818cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.698528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c65f9d-60cb-4b33-aab8-154c52818cbe admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.699255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c65f9d-60cb-4b33-aab8-154c52818cbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:01.699562 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c65f9d-60cb-4b33-aab8-154c52818cbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.701139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c65f9d-60cb-4b33-aab8-154c52818cbe admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.713050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1243/2420] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:56:01.717385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185218b7-a8e6-4e4d-9efe-66fb05951dc5 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.727573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185218b7-a8e6-4e4d-9efe-66fb05951dc5 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.732286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c343ae9-9eba-43ef-a70e-a1eb1e54258d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.732941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c343ae9-9eba-43ef-a70e-a1eb1e54258d admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.733234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c343ae9-9eba-43ef-a70e-a1eb1e54258d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.733811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c343ae9-9eba-43ef-a70e-a1eb1e54258d admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.734259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c343ae9-9eba-43ef-a70e-a1eb1e54258d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.736148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c343ae9-9eba-43ef-a70e-a1eb1e54258d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.761676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1178/2421] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:01 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:01.766323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c65f9d-60cb-4b33-aab8-154c52818cbe admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.776317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c65f9d-60cb-4b33-aab8-154c52818cbe admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.806511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:01.835238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684602ac-688b-4432-b214-0e12fd6414c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.835922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684602ac-688b-4432-b214-0e12fd6414c4 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.836213 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684602ac-688b-4432-b214-0e12fd6414c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.836495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684602ac-688b-4432-b214-0e12fd6414c4 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.837200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684602ac-688b-4432-b214-0e12fd6414c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.839227 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684602ac-688b-4432-b214-0e12fd6414c4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.864446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1244/2422] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:01 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:01.868766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c343ae9-9eba-43ef-a70e-a1eb1e54258d admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.884403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c343ae9-9eba-43ef-a70e-a1eb1e54258d admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.893148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35463b-e14c-41ee-a1c0-4efa2ca54e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.893786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35463b-e14c-41ee-a1c0-4efa2ca54e1d admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.894075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35463b-e14c-41ee-a1c0-4efa2ca54e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.894359 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35463b-e14c-41ee-a1c0-4efa2ca54e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.895321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35463b-e14c-41ee-a1c0-4efa2ca54e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.898019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35463b-e14c-41ee-a1c0-4efa2ca54e1d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.911489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1179/2423] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:01.915906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684602ac-688b-4432-b214-0e12fd6414c4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.926247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684602ac-688b-4432-b214-0e12fd6414c4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.930945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9c818-8ad8-4570-a2fd-791bd4ec5fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.931608 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c818-8ad8-4570-a2fd-791bd4ec5fcf admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.931891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c818-8ad8-4570-a2fd-791bd4ec5fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.932241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c818-8ad8-4570-a2fd-791bd4ec5fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.932907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9c818-8ad8-4570-a2fd-791bd4ec5fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.934593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9c818-8ad8-4570-a2fd-791bd4ec5fcf admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.946069 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1245/2424] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:01.950606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35463b-e14c-41ee-a1c0-4efa2ca54e1d admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.961303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35463b-e14c-41ee-a1c0-4efa2ca54e1d admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.966062 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd8b36-a8db-4b9f-9806-4b0c030a0d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.966975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8b36-a8db-4b9f-9806-4b0c030a0d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.967259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8b36-a8db-4b9f-9806-4b0c030a0d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.967536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8b36-a8db-4b9f-9806-4b0c030a0d8e admin admin] PATH_INFO: `/v3/projects/75d3e6b854204dcc88f79b0bcd561837/users/1b1517d4caf24ca69e70c5f354ea18cd/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.982893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd8b36-a8db-4b9f-9806-4b0c030a0d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=75d3e6b854204dcc88f79b0bcd561837, user_id=1b1517d4caf24ca69e70c5f354ea18cd, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.985620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd8b36-a8db-4b9f-9806-4b0c030a0d8e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.999867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1180/2425] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:01 2022] PUT /identity/v3/projects/75d3e6b854204dcc88f79b0bcd561837/users/1b1517d4caf24ca69e70c5f354ea18cd/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:02.068013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387a9a3-0283-4a62-8942-493456d37f1b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:02.068303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387a9a3-0283-4a62-8942-493456d37f1b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:02.068588 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387a9a3-0283-4a62-8942-493456d37f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:02.095370 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-8387a9a3-0283-4a62-8942-493456d37f1b None None] MFA Rules not processed for user `1b1517d4caf24ca69e70c5f354ea18cd`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:02.096593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8387a9a3-0283-4a62-8942-493456d37f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:02.167542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1246/2426] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:02.171953 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd8b36-a8db-4b9f-9806-4b0c030a0d8e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:02.264013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd8b36-a8db-4b9f-9806-4b0c030a0d8e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:02.270035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca829f-4fa2-46e3-a7d3-3254e497568e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:02.270733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca829f-4fa2-46e3-a7d3-3254e497568e admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:02.271018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca829f-4fa2-46e3-a7d3-3254e497568e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:02.271295 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca829f-4fa2-46e3-a7d3-3254e497568e admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:02.271997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca829f-4fa2-46e3-a7d3-3254e497568e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:02.273935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca829f-4fa2-46e3-a7d3-3254e497568e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:02.299885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1181/2427] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:02 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:02.305521 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387a9a3-0283-4a62-8942-493456d37f1b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:02.317183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387a9a3-0283-4a62-8942-493456d37f1b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:02.375308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991adb7-754d-4432-85c3-33f3a647ba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:02.375308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991adb7-754d-4432-85c3-33f3a647ba31 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:02.375308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991adb7-754d-4432-85c3-33f3a647ba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:02.375785 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991adb7-754d-4432-85c3-33f3a647ba31 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:02.376250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991adb7-754d-4432-85c3-33f3a647ba31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:02.378762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991adb7-754d-4432-85c3-33f3a647ba31 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:02.400845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1247/2428] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:02 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:56:02.404911 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca829f-4fa2-46e3-a7d3-3254e497568e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:02.415364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca829f-4fa2-46e3-a7d3-3254e497568e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:02.420299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391055a3-3d29-495d-82ad-5d1a1817a5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:02.420935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391055a3-3d29-495d-82ad-5d1a1817a5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:02.421223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391055a3-3d29-495d-82ad-5d1a1817a5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:02.421539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391055a3-3d29-495d-82ad-5d1a1817a5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:02.422844 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391055a3-3d29-495d-82ad-5d1a1817a5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:02.424520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391055a3-3d29-495d-82ad-5d1a1817a5a8 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:02.435294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1182/2429] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:02.440555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991adb7-754d-4432-85c3-33f3a647ba31 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:02.450735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991adb7-754d-4432-85c3-33f3a647ba31 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:02.455997 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495dc74-bf7e-455d-b31e-db5baf538e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:02.456876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495dc74-bf7e-455d-b31e-db5baf538e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:02.457154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495dc74-bf7e-455d-b31e-db5baf538e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:02.457483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495dc74-bf7e-455d-b31e-db5baf538e4a admin admin] PATH_INFO: `/v3/projects/bc8264fccd1e41cd9fa33a5819f6f008/users/d968b86a94a04c4bbb42f2859e4cc9c5/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:02.472119 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495dc74-bf7e-455d-b31e-db5baf538e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8264fccd1e41cd9fa33a5819f6f008, user_id=d968b86a94a04c4bbb42f2859e4cc9c5, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:02.474791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495dc74-bf7e-455d-b31e-db5baf538e4a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:02.488199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1248/2430] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:02 2022] PUT /identity/v3/projects/bc8264fccd1e41cd9fa33a5819f6f008/users/d968b86a94a04c4bbb42f2859e4cc9c5/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:02.492322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391055a3-3d29-495d-82ad-5d1a1817a5a8 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:02.502810 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391055a3-3d29-495d-82ad-5d1a1817a5a8 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:02.515094 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:02.559490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262b008-715a-42dc-ab87-c710c8006788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:02.560169 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262b008-715a-42dc-ab87-c710c8006788 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:02.560436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262b008-715a-42dc-ab87-c710c8006788 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:02.560713 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262b008-715a-42dc-ab87-c710c8006788 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:02.561459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262b008-715a-42dc-ab87-c710c8006788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:02.563620 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262b008-715a-42dc-ab87-c710c8006788 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:02.573505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1183/2431] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:02.577765 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495dc74-bf7e-455d-b31e-db5baf538e4a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:02.588269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495dc74-bf7e-455d-b31e-db5baf538e4a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:02.592938 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a95cc7-9153-4c37-af47-498efad10ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:02.593864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a95cc7-9153-4c37-af47-498efad10ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:02.594143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a95cc7-9153-4c37-af47-498efad10ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:02.594418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a95cc7-9153-4c37-af47-498efad10ac1 admin admin] PATH_INFO: `/v3/projects/bc8264fccd1e41cd9fa33a5819f6f008/users/d968b86a94a04c4bbb42f2859e4cc9c5/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:02.596426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a95cc7-9153-4c37-af47-498efad10ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8264fccd1e41cd9fa33a5819f6f008, user_id=d968b86a94a04c4bbb42f2859e4cc9c5, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:02.599092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a95cc7-9153-4c37-af47-498efad10ac1 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:02.601349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:02.612640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1249/2432] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:02 2022] PUT /identity/v3/projects/bc8264fccd1e41cd9fa33a5819f6f008/users/d968b86a94a04c4bbb42f2859e4cc9c5/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:02.623109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37152f16-74f7-4cdb-a009-375760902e13 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:02.623408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37152f16-74f7-4cdb-a009-375760902e13 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:02.623691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37152f16-74f7-4cdb-a009-375760902e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:02.650811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-37152f16-74f7-4cdb-a009-375760902e13 None None] MFA Rules not processed for user `d968b86a94a04c4bbb42f2859e4cc9c5`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:02.652212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-37152f16-74f7-4cdb-a009-375760902e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:02.720967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1184/2433] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:02.731278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a95cc7-9153-4c37-af47-498efad10ac1 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:02.732972 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-76a95cc7-9153-4c37-af47-498efad10ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:02.791100 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a95cc7-9153-4c37-af47-498efad10ac1 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:02.794766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d0a8a-67ba-46cb-a869-ca42082b0522 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:02.795416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d0a8a-67ba-46cb-a869-ca42082b0522 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:02.795701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d0a8a-67ba-46cb-a869-ca42082b0522 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:02.795987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d0a8a-67ba-46cb-a869-ca42082b0522 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:02.807886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d0a8a-67ba-46cb-a869-ca42082b0522 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:02.809468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d0a8a-67ba-46cb-a869-ca42082b0522 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:02.819174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1250/2434] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:03.200725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37152f16-74f7-4cdb-a009-375760902e13 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.267957 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37152f16-74f7-4cdb-a009-375760902e13 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.272399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.273266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.273577 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.273852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] PATH_INFO: `/v3/users/1b1517d4caf24ca69e70c5f354ea18cd` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.275040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b1517d4caf24ca69e70c5f354ea18cd)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.276932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.318044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:03.323617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b1517d4caf24ca69e70c5f354ea18cd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.332605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b1517d4caf24ca69e70c5f354ea18cd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.343132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b1517d4caf24ca69e70c5f354ea18cd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.356544 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b1517d4caf24ca69e70c5f354ea18cd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.357273 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b1517d4caf24ca69e70c5f354ea18cd'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.364775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1185/2435] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:03 2022] DELETE /identity/v3/users/1b1517d4caf24ca69e70c5f354ea18cd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:03.369502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57efa-857b-4f82-adcd-fdd46cffe6c3 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.369797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57efa-857b-4f82-adcd-fdd46cffe6c3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.370106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57efa-857b-4f82-adcd-fdd46cffe6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.388793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-eca57efa-857b-4f82-adcd-fdd46cffe6c3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:03.390030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-eca57efa-857b-4f82-adcd-fdd46cffe6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:03.450471 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1251/2436] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:03.463360 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.465677 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:03.528570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002aa8b7-3d34-4601-a941-c98dd4d32cd7 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.532462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a66b7f-09a5-4915-8ccc-d67b684bab65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.533213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a66b7f-09a5-4915-8ccc-d67b684bab65 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.533451 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a66b7f-09a5-4915-8ccc-d67b684bab65 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.533770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a66b7f-09a5-4915-8ccc-d67b684bab65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.543976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a66b7f-09a5-4915-8ccc-d67b684bab65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.545440 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a66b7f-09a5-4915-8ccc-d67b684bab65 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.555545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1186/2437] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:03.849132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca57efa-857b-4f82-adcd-fdd46cffe6c3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.850996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-eca57efa-857b-4f82-adcd-fdd46cffe6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:03.865040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca57efa-857b-4f82-adcd-fdd46cffe6c3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.871004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.872085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.872482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.872641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] PATH_INFO: `/v3/projects/75d3e6b854204dcc88f79b0bcd561837` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.873891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75d3e6b854204dcc88f79b0bcd561837)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.876400 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.925932 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75d3e6b854204dcc88f79b0bcd561837', 'request_id': 'req-6036f826-10c3-4755-88d2-ab8106885137'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.933232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75d3e6b854204dcc88f79b0bcd561837', 'request_id': 'req-6036f826-10c3-4755-88d2-ab8106885137'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.939335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75d3e6b854204dcc88f79b0bcd561837', 'request_id': 'req-6036f826-10c3-4755-88d2-ab8106885137'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.950460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] The token cache is being invalidate because project 75d3e6b854204dcc88f79b0bcd561837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:03.950750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.951394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.952957 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1252/2438] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:03 2022] DELETE /identity/v3/projects/75d3e6b854204dcc88f79b0bcd561837 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:03.956942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a66b7f-09a5-4915-8ccc-d67b684bab65 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.966931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a66b7f-09a5-4915-8ccc-d67b684bab65 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.023582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ecbf2-8bf5-41ad-ac0f-cdc882f8ac41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.024453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ecbf2-8bf5-41ad-ac0f-cdc882f8ac41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.024732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ecbf2-8bf5-41ad-ac0f-cdc882f8ac41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.025002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ecbf2-8bf5-41ad-ac0f-cdc882f8ac41 admin admin] PATH_INFO: `/v3/users/d968b86a94a04c4bbb42f2859e4cc9c5` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.026244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ecbf2-8bf5-41ad-ac0f-cdc882f8ac41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d968b86a94a04c4bbb42f2859e4cc9c5)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.028225 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ecbf2-8bf5-41ad-ac0f-cdc882f8ac41 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.076500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-682ecbf2-8bf5-41ad-ac0f-cdc882f8ac41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd968b86a94a04c4bbb42f2859e4cc9c5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:04.084384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-682ecbf2-8bf5-41ad-ac0f-cdc882f8ac41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd968b86a94a04c4bbb42f2859e4cc9c5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:04.091808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-682ecbf2-8bf5-41ad-ac0f-cdc882f8ac41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd968b86a94a04c4bbb42f2859e4cc9c5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:04.092816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:04.106400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-682ecbf2-8bf5-41ad-ac0f-cdc882f8ac41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd968b86a94a04c4bbb42f2859e4cc9c5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:04.107078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-682ecbf2-8bf5-41ad-ac0f-cdc882f8ac41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd968b86a94a04c4bbb42f2859e4cc9c5'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:04.113600 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1187/2439] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:03 2022] DELETE /identity/v3/users/d968b86a94a04c4bbb42f2859e4cc9c5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:04.389536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.391022 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:04.403917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036f826-10c3-4755-88d2-ab8106885137 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.423980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:04.466347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.467217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.467499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.467783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] PATH_INFO: `/v3/projects/bc8264fccd1e41cd9fa33a5819f6f008` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.469481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8264fccd1e41cd9fa33a5819f6f008)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.471440 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.519619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8264fccd1e41cd9fa33a5819f6f008', 'request_id': 'req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:04.532716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8264fccd1e41cd9fa33a5819f6f008', 'request_id': 'req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:04.538630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8264fccd1e41cd9fa33a5819f6f008', 'request_id': 'req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:04.550195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] The token cache is being invalidate because project bc8264fccd1e41cd9fa33a5819f6f008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:04.550489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:04.551149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-147695d0-4b6e-4ad7-ad79-74b30f0d52c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:04.552695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1253/2440] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:04 2022] DELETE /identity/v3/projects/bc8264fccd1e41cd9fa33a5819f6f008 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:04.559270 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce27ec0-9a99-42a9-8839-55a8aa958b24 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.559775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce27ec0-9a99-42a9-8839-55a8aa958b24 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.559873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce27ec0-9a99-42a9-8839-55a8aa958b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.580059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-bce27ec0-9a99-42a9-8839-55a8aa958b24 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:04.581351 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bce27ec0-9a99-42a9-8839-55a8aa958b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:04.644590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1188/2441] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:04.652802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d8a7d-deb5-4d5e-a7d3-c0ac23de97ee None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.653091 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d8a7d-deb5-4d5e-a7d3-c0ac23de97ee None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.653394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d8a7d-deb5-4d5e-a7d3-c0ac23de97ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.672617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-139d8a7d-deb5-4d5e-a7d3-c0ac23de97ee None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:04.673864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-139d8a7d-deb5-4d5e-a7d3-c0ac23de97ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:04.682446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1254/2442] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:04.686907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce27ec0-9a99-42a9-8839-55a8aa958b24 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.696800 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce27ec0-9a99-42a9-8839-55a8aa958b24 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.701478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3731ab-414a-4401-a963-f939bb18f056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.702140 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3731ab-414a-4401-a963-f939bb18f056 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.702420 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3731ab-414a-4401-a963-f939bb18f056 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.702692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3731ab-414a-4401-a963-f939bb18f056 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.703973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3731ab-414a-4401-a963-f939bb18f056 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:04.704312 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3731ab-414a-4401-a963-f939bb18f056 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.705992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3731ab-414a-4401-a963-f939bb18f056 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.718066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1189/2443] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:56:04.723359 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d8a7d-deb5-4d5e-a7d3-c0ac23de97ee None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.732940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d8a7d-deb5-4d5e-a7d3-c0ac23de97ee None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.738077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8f183-62a8-4319-9199-466938be75c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.738733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8f183-62a8-4319-9199-466938be75c4 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.739015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8f183-62a8-4319-9199-466938be75c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.739290 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8f183-62a8-4319-9199-466938be75c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.739988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8f183-62a8-4319-9199-466938be75c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.741913 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8f183-62a8-4319-9199-466938be75c4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.770921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1255/2444] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:04 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:04.775801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3731ab-414a-4401-a963-f939bb18f056 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.786606 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3731ab-414a-4401-a963-f939bb18f056 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.844365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae493a8c-bcc5-4753-bc54-06631ff81b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.844997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae493a8c-bcc5-4753-bc54-06631ff81b44 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.845275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae493a8c-bcc5-4753-bc54-06631ff81b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.845579 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae493a8c-bcc5-4753-bc54-06631ff81b44 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.846260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae493a8c-bcc5-4753-bc54-06631ff81b44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.848151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae493a8c-bcc5-4753-bc54-06631ff81b44 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.867846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1190/2445] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:04 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:04.872153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8f183-62a8-4319-9199-466938be75c4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.883267 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8f183-62a8-4319-9199-466938be75c4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.889619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd37131-4949-4e76-b540-177e16121893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.889619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd37131-4949-4e76-b540-177e16121893 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.890015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd37131-4949-4e76-b540-177e16121893 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.890015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd37131-4949-4e76-b540-177e16121893 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.891315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd37131-4949-4e76-b540-177e16121893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.892912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd37131-4949-4e76-b540-177e16121893 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.903674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1256/2446] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:04.909510 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae493a8c-bcc5-4753-bc54-06631ff81b44 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.919463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae493a8c-bcc5-4753-bc54-06631ff81b44 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.924171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbb586-5654-40a1-af1d-420a73adee47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.924827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbb586-5654-40a1-af1d-420a73adee47 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.925105 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbb586-5654-40a1-af1d-420a73adee47 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.925400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbb586-5654-40a1-af1d-420a73adee47 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.926127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbb586-5654-40a1-af1d-420a73adee47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.927733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbb586-5654-40a1-af1d-420a73adee47 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.938348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1191/2447] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:04.942643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd37131-4949-4e76-b540-177e16121893 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.952612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd37131-4949-4e76-b540-177e16121893 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.958370 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a7414-fe24-4359-9401-cb78fc5ea75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.959283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a7414-fe24-4359-9401-cb78fc5ea75f admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.959571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a7414-fe24-4359-9401-cb78fc5ea75f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.959848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a7414-fe24-4359-9401-cb78fc5ea75f admin admin] PATH_INFO: `/v3/projects/3dcf50fca2134c84812de75fed624869/users/c0bbce46598c465bb29cb0353f10e3eb/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.975994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a7414-fe24-4359-9401-cb78fc5ea75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dcf50fca2134c84812de75fed624869, user_id=c0bbce46598c465bb29cb0353f10e3eb, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.978942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a7414-fe24-4359-9401-cb78fc5ea75f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.993204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1257/2448] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:04 2022] PUT /identity/v3/projects/3dcf50fca2134c84812de75fed624869/users/c0bbce46598c465bb29cb0353f10e3eb/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:04.998370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a382af8-af4f-40e5-95f1-a6b6ea604121 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.998667 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a382af8-af4f-40e5-95f1-a6b6ea604121 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.998954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a382af8-af4f-40e5-95f1-a6b6ea604121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:05.017692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-8a382af8-af4f-40e5-95f1-a6b6ea604121 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:05.018905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8a382af8-af4f-40e5-95f1-a6b6ea604121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:05.060718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:05.080718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1192/2449] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:05.090275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a7414-fe24-4359-9401-cb78fc5ea75f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:05.091962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1d2a7414-fe24-4359-9401-cb78fc5ea75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:05.146116 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:05.151443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a7414-fe24-4359-9401-cb78fc5ea75f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:05.157109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f42358-0268-48c2-af8a-3f44a5185dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:05.157954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42358-0268-48c2-af8a-3f44a5185dae service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:05.158545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42358-0268-48c2-af8a-3f44a5185dae service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:05.159009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42358-0268-48c2-af8a-3f44a5185dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:05.170140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f42358-0268-48c2-af8a-3f44a5185dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:05.171489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f42358-0268-48c2-af8a-3f44a5185dae service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:05.181157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1258/2450] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:09.301840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796def08-2fed-4b86-a740-e85155f8b29d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:09.302335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796def08-2fed-4b86-a740-e85155f8b29d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:09.302531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796def08-2fed-4b86-a740-e85155f8b29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:09.334037 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-796def08-2fed-4b86-a740-e85155f8b29d None None] MFA Rules not processed for user `c0bbce46598c465bb29cb0353f10e3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:09.335293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-796def08-2fed-4b86-a740-e85155f8b29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:09.426224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1193/2451] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:09.431246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f42358-0268-48c2-af8a-3f44a5185dae service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:09.441804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f42358-0268-48c2-af8a-3f44a5185dae service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:09.447763 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56d5c4-0645-481d-a1e3-5508f65511f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:09.448747 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56d5c4-0645-481d-a1e3-5508f65511f5 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:09.448882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56d5c4-0645-481d-a1e3-5508f65511f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:09.449045 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56d5c4-0645-481d-a1e3-5508f65511f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:09.450235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56d5c4-0645-481d-a1e3-5508f65511f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:09.453916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56d5c4-0645-481d-a1e3-5508f65511f5 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:09.485248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1259/2452] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:09 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:09.490046 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796def08-2fed-4b86-a740-e85155f8b29d None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:09.503015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796def08-2fed-4b86-a740-e85155f8b29d None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:09.573571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fb176-136c-48af-a687-f0c70cdf9938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:09.574320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb176-136c-48af-a687-f0c70cdf9938 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:09.574672 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb176-136c-48af-a687-f0c70cdf9938 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:09.575020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb176-136c-48af-a687-f0c70cdf9938 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:09.575793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fb176-136c-48af-a687-f0c70cdf9938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:09.578134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fb176-136c-48af-a687-f0c70cdf9938 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:09.604327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1194/2453] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:09 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:09.606115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56d5c4-0645-481d-a1e3-5508f65511f5 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:09.618884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56d5c4-0645-481d-a1e3-5508f65511f5 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:09.624416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc80fd-4032-45b5-83a9-8671bf9fce72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:09.625145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc80fd-4032-45b5-83a9-8671bf9fce72 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:09.625470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc80fd-4032-45b5-83a9-8671bf9fce72 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:09.625748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc80fd-4032-45b5-83a9-8671bf9fce72 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:09.626614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc80fd-4032-45b5-83a9-8671bf9fce72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:09.628504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc80fd-4032-45b5-83a9-8671bf9fce72 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:09.640611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1260/2454] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:09.644717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fb176-136c-48af-a687-f0c70cdf9938 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:09.657038 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fb176-136c-48af-a687-f0c70cdf9938 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:09.662867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8555a1-79d2-489c-9e1f-4657759c986a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:09.664115 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8555a1-79d2-489c-9e1f-4657759c986a admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:09.664275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8555a1-79d2-489c-9e1f-4657759c986a admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:09.664826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8555a1-79d2-489c-9e1f-4657759c986a admin admin] PATH_INFO: `/v3/projects/8631943115184923a07591723a58833e/users/80b9b704671047439993d6d91170b1b2/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:09.685124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8555a1-79d2-489c-9e1f-4657759c986a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8631943115184923a07591723a58833e, user_id=80b9b704671047439993d6d91170b1b2, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:09.689216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8555a1-79d2-489c-9e1f-4657759c986a admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:09.703853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1195/2455] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:09 2022] PUT /identity/v3/projects/8631943115184923a07591723a58833e/users/80b9b704671047439993d6d91170b1b2/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:09.708340 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc80fd-4032-45b5-83a9-8671bf9fce72 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:09.720243 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc80fd-4032-45b5-83a9-8671bf9fce72 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:09.793652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe163b8d-eb49-473d-91ed-5611a15e0ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:09.794341 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163b8d-eb49-473d-91ed-5611a15e0ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:09.794627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163b8d-eb49-473d-91ed-5611a15e0ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:09.794905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163b8d-eb49-473d-91ed-5611a15e0ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:09.795629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe163b8d-eb49-473d-91ed-5611a15e0ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:09.797296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe163b8d-eb49-473d-91ed-5611a15e0ac8 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:09.810518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1261/2456] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:09.811890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8555a1-79d2-489c-9e1f-4657759c986a admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:09.822871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8555a1-79d2-489c-9e1f-4657759c986a admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:09.827647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8229e-6330-4e63-be79-93d675916804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:09.829061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8229e-6330-4e63-be79-93d675916804 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:09.829345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8229e-6330-4e63-be79-93d675916804 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:09.829672 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8229e-6330-4e63-be79-93d675916804 admin admin] PATH_INFO: `/v3/projects/8631943115184923a07591723a58833e/users/80b9b704671047439993d6d91170b1b2/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:09.831643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8229e-6330-4e63-be79-93d675916804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8631943115184923a07591723a58833e, user_id=80b9b704671047439993d6d91170b1b2, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:09.834340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8229e-6330-4e63-be79-93d675916804 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:09.845721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1196/2457] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:09 2022] PUT /identity/v3/projects/8631943115184923a07591723a58833e/users/80b9b704671047439993d6d91170b1b2/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:11.910565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe163b8d-eb49-473d-91ed-5611a15e0ac8 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:11.913047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fe163b8d-eb49-473d-91ed-5611a15e0ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:12.013117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe163b8d-eb49-473d-91ed-5611a15e0ac8 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:12.114474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a6f8b-ba64-4aa1-9eed-54185bc35b14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:12.115377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a6f8b-ba64-4aa1-9eed-54185bc35b14 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:12.115782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a6f8b-ba64-4aa1-9eed-54185bc35b14 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:12.116310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a6f8b-ba64-4aa1-9eed-54185bc35b14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:12.122859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b57a6f8b-ba64-4aa1-9eed-54185bc35b14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:12.181622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:12.258692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a6f8b-ba64-4aa1-9eed-54185bc35b14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:12.260472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a6f8b-ba64-4aa1-9eed-54185bc35b14 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:12.271931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1262/2458] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:15.504226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ec82d-724b-4fe3-b538-fed365862366 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:15.504760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ec82d-724b-4fe3-b538-fed365862366 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:15.504916 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ec82d-724b-4fe3-b538-fed365862366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:15.539146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-6d5ec82d-724b-4fe3-b538-fed365862366 None None] MFA Rules not processed for user `80b9b704671047439993d6d91170b1b2`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:15.557827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6d5ec82d-724b-4fe3-b538-fed365862366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:15.598969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:15.684335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1197/2459] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:15.700324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a6f8b-ba64-4aa1-9eed-54185bc35b14 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:15.702702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b57a6f8b-ba64-4aa1-9eed-54185bc35b14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:15.786897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a6f8b-ba64-4aa1-9eed-54185bc35b14 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:15.801537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496fad9-f53d-49de-a4aa-6cec40020e5a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82e1192b2d554199a08debb4aa392479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:15.802367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496fad9-f53d-49de-a4aa-6cec40020e5a service glance] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:15.802746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496fad9-f53d-49de-a4aa-6cec40020e5a service glance] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:15.803503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496fad9-f53d-49de-a4aa-6cec40020e5a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:15.901488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496fad9-f53d-49de-a4aa-6cec40020e5a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:15.903005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496fad9-f53d-49de-a4aa-6cec40020e5a service glance] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:15.924013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1263/2460] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:16.077612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60618482-57a0-4a28-99af-2d77de6cfb25 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:16.078012 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60618482-57a0-4a28-99af-2d77de6cfb25 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:16.078208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60618482-57a0-4a28-99af-2d77de6cfb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:16.100514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-60618482-57a0-4a28-99af-2d77de6cfb25 None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:16.101917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-60618482-57a0-4a28-99af-2d77de6cfb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:16.170206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1198/2461] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:56:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:16.236271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496fad9-f53d-49de-a4aa-6cec40020e5a service glance] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:16.238030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0496fad9-f53d-49de-a4aa-6cec40020e5a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:16.313825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496fad9-f53d-49de-a4aa-6cec40020e5a service glance] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:16.318933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec680e-5e4f-4546-a020-7b8784b22557 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:16.319591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec680e-5e4f-4546-a020-7b8784b22557 service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:16.319873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec680e-5e4f-4546-a020-7b8784b22557 service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:16.320150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec680e-5e4f-4546-a020-7b8784b22557 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:16.328945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec680e-5e4f-4546-a020-7b8784b22557 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:16.330618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec680e-5e4f-4546-a020-7b8784b22557 service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:16.338936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1264/2462] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:56:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:56:19.815087 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e63e2-aeb6-4175-98a7-d7b81e1feb63 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:19.815659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e63e2-aeb6-4175-98a7-d7b81e1feb63 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:19.815659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e63e2-aeb6-4175-98a7-d7b81e1feb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:19.840533 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-452e63e2-aeb6-4175-98a7-d7b81e1feb63 None None] MFA Rules not processed for user `cbcb993e2d1842389c5862f0fdb7d4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:19.841809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-452e63e2-aeb6-4175-98a7-d7b81e1feb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:19.850432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1199/2463] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:26.335579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:26.544301 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60739151-233b-40d9-aeb7-bcbcaedc8c5b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:26.544695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60739151-233b-40d9-aeb7-bcbcaedc8c5b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:26.545092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60739151-233b-40d9-aeb7-bcbcaedc8c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:26.563677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-60739151-233b-40d9-aeb7-bcbcaedc8c5b None None] MFA Rules not processed for user `c0bbce46598c465bb29cb0353f10e3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:26.564976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-60739151-233b-40d9-aeb7-bcbcaedc8c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:26.572721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1265/2464] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:56:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:28.461716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209e6c2-92b0-412d-a486-aa3c0105aa1d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:28.462545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209e6c2-92b0-412d-a486-aa3c0105aa1d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:28.462831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209e6c2-92b0-412d-a486-aa3c0105aa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:28.483867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-5209e6c2-92b0-412d-a486-aa3c0105aa1d None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:28.485164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5209e6c2-92b0-412d-a486-aa3c0105aa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:28.494147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1200/2465] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:28.507007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60739151-233b-40d9-aeb7-bcbcaedc8c5b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:28.519697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60739151-233b-40d9-aeb7-bcbcaedc8c5b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:28.524732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642329b-d55d-491c-bcee-e1645093f2c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:28.525439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642329b-d55d-491c-bcee-e1645093f2c7 service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:28.525728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642329b-d55d-491c-bcee-e1645093f2c7 service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:28.526008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642329b-d55d-491c-bcee-e1645093f2c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:28.534844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642329b-d55d-491c-bcee-e1645093f2c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:28.536463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642329b-d55d-491c-bcee-e1645093f2c7 service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:28.545280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1266/2466] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:56:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:56:38.475685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=52 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:38.541819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=9 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:03.357778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209e6c2-92b0-412d-a486-aa3c0105aa1d None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:03.436455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209e6c2-92b0-412d-a486-aa3c0105aa1d None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:03.440113 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177f4e2-b304-4386-9383-1b1eda20e30d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:03.440765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f4e2-b304-4386-9383-1b1eda20e30d service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:03.441048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f4e2-b304-4386-9383-1b1eda20e30d service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:03.441324 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f4e2-b304-4386-9383-1b1eda20e30d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:03.473281 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177f4e2-b304-4386-9383-1b1eda20e30d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:03.474752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177f4e2-b304-4386-9383-1b1eda20e30d service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:03.484559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1201/2467] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:57:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:57:13.480312 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=44 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:18.399407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f6cf5-fb7b-453a-9917-7ea42358a000 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:18.400190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f6cf5-fb7b-453a-9917-7ea42358a000 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:18.400190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f6cf5-fb7b-453a-9917-7ea42358a000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:18.431435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-519f6cf5-fb7b-453a-9917-7ea42358a000 None None] MFA Rules not processed for user `af989ff6dc024ff19023e6f633915ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:18.432647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-519f6cf5-fb7b-453a-9917-7ea42358a000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:18.505835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1267/2468] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:57:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:28.497917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=29 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:34.183018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b3029-b1db-439a-9b9f-170d30a58487 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:34.183561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b3029-b1db-439a-9b9f-170d30a58487 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:34.183561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b3029-b1db-439a-9b9f-170d30a58487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:34.237299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-904b3029-b1db-439a-9b9f-170d30a58487 None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:34.238556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-904b3029-b1db-439a-9b9f-170d30a58487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:34.255601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1202/2469] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:57:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:34.303662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f6cf5-fb7b-453a-9917-7ea42358a000 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:34.317737 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f6cf5-fb7b-453a-9917-7ea42358a000 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:34.323482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6693d81-7481-44e7-911b-94347af8a2d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:34.324155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6693d81-7481-44e7-911b-94347af8a2d4 service swift] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:34.324443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6693d81-7481-44e7-911b-94347af8a2d4 service swift] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:34.324709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6693d81-7481-44e7-911b-94347af8a2d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:34.337819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6693d81-7481-44e7-911b-94347af8a2d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:34.340333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6693d81-7481-44e7-911b-94347af8a2d4 service swift] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:34.355627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1268/2470] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:57:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:57:44.247148 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=10 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:44.349882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=6 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:48.008534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b3029-b1db-439a-9b9f-170d30a58487 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:48.010131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-904b3029-b1db-439a-9b9f-170d30a58487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:48.112873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b3029-b1db-439a-9b9f-170d30a58487 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:48.118145 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd8c7a-a836-4836-80a9-d589be8368cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:48.118812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd8c7a-a836-4836-80a9-d589be8368cc service placement] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:48.119097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd8c7a-a836-4836-80a9-d589be8368cc service placement] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:48.119378 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd8c7a-a836-4836-80a9-d589be8368cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:48.121417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c7dd8c7a-a836-4836-80a9-d589be8368cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:48.134288 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd8c7a-a836-4836-80a9-d589be8368cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:48.135883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd8c7a-a836-4836-80a9-d589be8368cc service placement] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:48.146788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1203/2471] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:57:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:57:52.255494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbee8-bda6-443e-b39d-c3d6f958be39 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:52.256065 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbee8-bda6-443e-b39d-c3d6f958be39 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:52.256065 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbee8-bda6-443e-b39d-c3d6f958be39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:52.274306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-d0dcbee8-bda6-443e-b39d-c3d6f958be39 None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:52.275560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d0dcbee8-bda6-443e-b39d-c3d6f958be39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:52.283096 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1269/2472] 10.222.0.70 () {58 vars in 1009 bytes} [Tue Jul 19 20:57:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:52.295056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd8c7a-a836-4836-80a9-d589be8368cc service placement] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:52.307003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd8c7a-a836-4836-80a9-d589be8368cc service placement] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:52.312747 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d731d00-0394-4ace-aadb-06430786aef4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:52.313430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d731d00-0394-4ace-aadb-06430786aef4 service swift] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:52.313724 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d731d00-0394-4ace-aadb-06430786aef4 service swift] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:52.314002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d731d00-0394-4ace-aadb-06430786aef4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:52.322856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d731d00-0394-4ace-aadb-06430786aef4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:52.324436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d731d00-0394-4ace-aadb-06430786aef4 service swift] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:52.333876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1204/2473] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:57:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:58:02.268942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:02.330009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=47 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:15.901638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bb7a1-91ac-4032-8619-8f821143fce1 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:15.902214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bb7a1-91ac-4032-8619-8f821143fce1 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:15.902214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bb7a1-91ac-4032-8619-8f821143fce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:15.923024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-051bb7a1-91ac-4032-8619-8f821143fce1 None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:58:15.924359 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-051bb7a1-91ac-4032-8619-8f821143fce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:15.931883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1270/2474] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jul 19 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:58:15.944429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d731d00-0394-4ace-aadb-06430786aef4 service swift] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:15.955425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d731d00-0394-4ace-aadb-06430786aef4 service swift] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:15.960570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff84490-8bda-4674-846e-4403d529fc08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:15.961232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff84490-8bda-4674-846e-4403d529fc08 service swift] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:15.961557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff84490-8bda-4674-846e-4403d529fc08 service swift] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:15.961881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff84490-8bda-4674-846e-4403d529fc08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:15.971752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff84490-8bda-4674-846e-4403d529fc08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:15.973347 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff84490-8bda-4674-846e-4403d529fc08 service swift] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:15.984177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1205/2475] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 20:58:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:58:25.915300 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:25.979332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=6 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:57.003988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bb7a1-91ac-4032-8619-8f821143fce1 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:57.006309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-051bb7a1-91ac-4032-8619-8f821143fce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:57.048894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bb7a1-91ac-4032-8619-8f821143fce1 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:57.054182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45086b-a96b-4ad1-a404-3c846148426b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:57.055266 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45086b-a96b-4ad1-a404-3c846148426b service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:57.055773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45086b-a96b-4ad1-a404-3c846148426b service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:57.056289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45086b-a96b-4ad1-a404-3c846148426b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:57.059696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-cf45086b-a96b-4ad1-a404-3c846148426b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:57.177173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45086b-a96b-4ad1-a404-3c846148426b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:57.178894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45086b-a96b-4ad1-a404-3c846148426b service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:57.219303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1271/2476] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:58:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:03.012729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff84490-8bda-4674-846e-4403d529fc08 service swift] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:03.022275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff84490-8bda-4674-846e-4403d529fc08 service swift] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:03.026089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0c5fc-1a82-4e60-855b-ed2effa53a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:03.026732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c5fc-1a82-4e60-855b-ed2effa53a9d service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:03.027022 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c5fc-1a82-4e60-855b-ed2effa53a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:03.027302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c5fc-1a82-4e60-855b-ed2effa53a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:03.029327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-add0c5fc-1a82-4e60-855b-ed2effa53a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:03.119243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0c5fc-1a82-4e60-855b-ed2effa53a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:03.120732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0c5fc-1a82-4e60-855b-ed2effa53a9d service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:03.150537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1206/2477] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:07.055305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45086b-a96b-4ad1-a404-3c846148426b service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:07.067483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45086b-a96b-4ad1-a404-3c846148426b service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:07.073980 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542d565-0602-491d-a4bd-e09656d0130b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:07.075115 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542d565-0602-491d-a4bd-e09656d0130b service placement] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:07.075448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542d565-0602-491d-a4bd-e09656d0130b service placement] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:07.076807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542d565-0602-491d-a4bd-e09656d0130b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:07.078833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0542d565-0602-491d-a4bd-e09656d0130b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:07.092831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542d565-0602-491d-a4bd-e09656d0130b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:07.094560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542d565-0602-491d-a4bd-e09656d0130b service placement] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:07.108355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1272/2478] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:07.138220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0c5fc-1a82-4e60-855b-ed2effa53a9d service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:07.140238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-add0c5fc-1a82-4e60-855b-ed2effa53a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:07.158700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0c5fc-1a82-4e60-855b-ed2effa53a9d service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:07.164156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fc0fe-aa11-41a2-bdce-14f86742fa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:07.165234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fc0fe-aa11-41a2-bdce-14f86742fa64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:07.165513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fc0fe-aa11-41a2-bdce-14f86742fa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:07.165931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fc0fe-aa11-41a2-bdce-14f86742fa64 admin admin] PATH_INFO: `/v3/users/cbcb993e2d1842389c5862f0fdb7d4d7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:07.167704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fc0fe-aa11-41a2-bdce-14f86742fa64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbcb993e2d1842389c5862f0fdb7d4d7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:07.170366 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fc0fe-aa11-41a2-bdce-14f86742fa64 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:07.243403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cf7fc0fe-aa11-41a2-bdce-14f86742fa64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbcb993e2d1842389c5862f0fdb7d4d7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:07.253776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cf7fc0fe-aa11-41a2-bdce-14f86742fa64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbcb993e2d1842389c5862f0fdb7d4d7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:07.263530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cf7fc0fe-aa11-41a2-bdce-14f86742fa64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbcb993e2d1842389c5862f0fdb7d4d7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:07.290779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cf7fc0fe-aa11-41a2-bdce-14f86742fa64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbcb993e2d1842389c5862f0fdb7d4d7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:07.291972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cf7fc0fe-aa11-41a2-bdce-14f86742fa64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbcb993e2d1842389c5862f0fdb7d4d7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:07.312645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1207/2479] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:59:07 2022] DELETE /identity/v3/users/cbcb993e2d1842389c5862f0fdb7d4d7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:07.503967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542d565-0602-491d-a4bd-e09656d0130b service placement] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:07.521133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542d565-0602-491d-a4bd-e09656d0130b service placement] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:07.584126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:07.584931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:07.585222 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:07.585522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] PATH_INFO: `/v3/projects/a35c1526c58f4c2b9b44a4de6c945846` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:07.586829 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a35c1526c58f4c2b9b44a4de6c945846)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:07.588762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:07.626143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:07.637166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a35c1526c58f4c2b9b44a4de6c945846', 'request_id': 'req-5ac9954b-127a-4670-a286-5e705e1f9e65'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:07.645995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a35c1526c58f4c2b9b44a4de6c945846', 'request_id': 'req-5ac9954b-127a-4670-a286-5e705e1f9e65'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:07.655316 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a35c1526c58f4c2b9b44a4de6c945846', 'request_id': 'req-5ac9954b-127a-4670-a286-5e705e1f9e65'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:07.669268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] The token cache is being invalidate because project a35c1526c58f4c2b9b44a4de6c945846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:59:07.669632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:07.670229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5ac9954b-127a-4670-a286-5e705e1f9e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:07.671758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1273/2480] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:59:07 2022] DELETE /identity/v3/projects/a35c1526c58f4c2b9b44a4de6c945846 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:07.677777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac63a34-d833-4b58-82a8-97d6c0870d95 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:07.678127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac63a34-d833-4b58-82a8-97d6c0870d95 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:07.678455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac63a34-d833-4b58-82a8-97d6c0870d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:07.736301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:07.739892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-dac63a34-d833-4b58-82a8-97d6c0870d95 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:07.741722 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-dac63a34-d833-4b58-82a8-97d6c0870d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:07.803237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1208/2481] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:07.816670 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced5155-68be-4d99-b478-09d4215f74b3 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:07.817236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced5155-68be-4d99-b478-09d4215f74b3 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:07.817530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced5155-68be-4d99-b478-09d4215f74b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:07.840934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-6ced5155-68be-4d99-b478-09d4215f74b3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:07.842122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6ced5155-68be-4d99-b478-09d4215f74b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:07.850658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1274/2482] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:07.855043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac63a34-d833-4b58-82a8-97d6c0870d95 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:07.865935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac63a34-d833-4b58-82a8-97d6c0870d95 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:07.870579 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08790a41-82a7-40a8-ae55-8b3e21d57c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:07.871243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08790a41-82a7-40a8-ae55-8b3e21d57c78 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:07.871527 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08790a41-82a7-40a8-ae55-8b3e21d57c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:07.871833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08790a41-82a7-40a8-ae55-8b3e21d57c78 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:07.873054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08790a41-82a7-40a8-ae55-8b3e21d57c78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:59:07.873372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08790a41-82a7-40a8-ae55-8b3e21d57c78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:07.875007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08790a41-82a7-40a8-ae55-8b3e21d57c78 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:07.888987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1209/2483] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:59:07.893501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced5155-68be-4d99-b478-09d4215f74b3 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:07.903445 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced5155-68be-4d99-b478-09d4215f74b3 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:07.908266 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c242c-e87c-449c-8774-2ee535b0d5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:07.908924 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c242c-e87c-449c-8774-2ee535b0d5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:07.909209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c242c-e87c-449c-8774-2ee535b0d5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:07.909506 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c242c-e87c-449c-8774-2ee535b0d5e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:07.910218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c242c-e87c-449c-8774-2ee535b0d5e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:07.912113 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c242c-e87c-449c-8774-2ee535b0d5e1 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:07.952734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1275/2484] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:59:07 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:07.955437 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08790a41-82a7-40a8-ae55-8b3e21d57c78 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:07.965987 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08790a41-82a7-40a8-ae55-8b3e21d57c78 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:08.064918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f51b4-e75c-4138-bfad-001254ce6f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:08.065651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f51b4-e75c-4138-bfad-001254ce6f5c admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:08.065940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f51b4-e75c-4138-bfad-001254ce6f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:08.066221 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f51b4-e75c-4138-bfad-001254ce6f5c admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:08.066927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f51b4-e75c-4138-bfad-001254ce6f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:08.069149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f51b4-e75c-4138-bfad-001254ce6f5c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:08.094217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1210/2485] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:59:07 2022] POST /identity/v3/users => generated 284 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:08.099504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c242c-e87c-449c-8774-2ee535b0d5e1 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:08.111952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c242c-e87c-449c-8774-2ee535b0d5e1 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:08.117309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f32ae-a61c-45f8-931e-4e3177c95c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:08.118052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f32ae-a61c-45f8-931e-4e3177c95c69 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:08.118306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f32ae-a61c-45f8-931e-4e3177c95c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:08.118581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f32ae-a61c-45f8-931e-4e3177c95c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:08.119313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f32ae-a61c-45f8-931e-4e3177c95c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:08.120903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f32ae-a61c-45f8-931e-4e3177c95c69 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:08.132085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1276/2486] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:08.136391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f51b4-e75c-4138-bfad-001254ce6f5c admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:08.147806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f51b4-e75c-4138-bfad-001254ce6f5c admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:08.158204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40155ee-576d-40f9-ba66-7842b080e8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:08.158894 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40155ee-576d-40f9-ba66-7842b080e8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:08.159177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40155ee-576d-40f9-ba66-7842b080e8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:08.159455 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40155ee-576d-40f9-ba66-7842b080e8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:08.160189 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40155ee-576d-40f9-ba66-7842b080e8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:08.161933 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40155ee-576d-40f9-ba66-7842b080e8d2 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:08.173624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1211/2487] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:08.179742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f32ae-a61c-45f8-931e-4e3177c95c69 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:08.194033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f32ae-a61c-45f8-931e-4e3177c95c69 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:08.199063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bda8da-dacc-4240-9879-5b005957193c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:08.200006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bda8da-dacc-4240-9879-5b005957193c admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:08.200293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bda8da-dacc-4240-9879-5b005957193c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:08.200610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bda8da-dacc-4240-9879-5b005957193c admin admin] PATH_INFO: `/v3/projects/bd6b521e4fa241a984857a94fbde044e/users/67085b76872042cca7a63fbb842b02b0/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:08.217602 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bda8da-dacc-4240-9879-5b005957193c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd6b521e4fa241a984857a94fbde044e, user_id=67085b76872042cca7a63fbb842b02b0, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:08.220258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bda8da-dacc-4240-9879-5b005957193c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:08.234394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1277/2488] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:08 2022] PUT /identity/v3/projects/bd6b521e4fa241a984857a94fbde044e/users/67085b76872042cca7a63fbb842b02b0/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:08.249622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc6a30-ac6d-4508-a15d-4ffde36f7bd0 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:08.249622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc6a30-ac6d-4508-a15d-4ffde36f7bd0 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:08.249622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc6a30-ac6d-4508-a15d-4ffde36f7bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:08.276265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-8acc6a30-ac6d-4508-a15d-4ffde36f7bd0 None None] MFA Rules not processed for user `67085b76872042cca7a63fbb842b02b0`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:08.277529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8acc6a30-ac6d-4508-a15d-4ffde36f7bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:08.347021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1212/2489] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:08.359054 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bda8da-dacc-4240-9879-5b005957193c admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:08.360931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-62bda8da-dacc-4240-9879-5b005957193c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:08.466542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc6a30-ac6d-4508-a15d-4ffde36f7bd0 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:08.467995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8acc6a30-ac6d-4508-a15d-4ffde36f7bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:08.477102 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:08.479491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bda8da-dacc-4240-9879-5b005957193c admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:08.486660 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81b014-044e-4712-b593-7ecbf6f9bdf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:08.487857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81b014-044e-4712-b593-7ecbf6f9bdf4 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:08.488306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc6a30-ac6d-4508-a15d-4ffde36f7bd0 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:08.488791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81b014-044e-4712-b593-7ecbf6f9bdf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:08.489482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81b014-044e-4712-b593-7ecbf6f9bdf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:08.500861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeebc7c-b9fa-4942-8e08-4d350767010d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:08.502128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeebc7c-b9fa-4942-8e08-4d350767010d service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:08.502672 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeebc7c-b9fa-4942-8e08-4d350767010d service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:08.503156 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeebc7c-b9fa-4942-8e08-4d350767010d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:08.505463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81b014-044e-4712-b593-7ecbf6f9bdf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:08.505819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-eaeebc7c-b9fa-4942-8e08-4d350767010d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:08.508443 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81b014-044e-4712-b593-7ecbf6f9bdf4 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:08.524158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1278/2490] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:08.630441 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeebc7c-b9fa-4942-8e08-4d350767010d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:08.633281 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeebc7c-b9fa-4942-8e08-4d350767010d service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:08.657918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1213/2491] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:59:17.183358 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81b014-044e-4712-b593-7ecbf6f9bdf4 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:17.240547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:17.278472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81b014-044e-4712-b593-7ecbf6f9bdf4 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:17.284575 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:17.285933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.286450 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.286852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] PATH_INFO: `/v3/users/67085b76872042cca7a63fbb842b02b0` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.288338 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67085b76872042cca7a63fbb842b02b0)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:17.290346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:17.347055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67085b76872042cca7a63fbb842b02b0'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.359887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67085b76872042cca7a63fbb842b02b0'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.368575 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67085b76872042cca7a63fbb842b02b0'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.369281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67085b76872042cca7a63fbb842b02b0'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.378971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67085b76872042cca7a63fbb842b02b0'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.387716 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1279/2492] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:59:17 2022] DELETE /identity/v3/users/67085b76872042cca7a63fbb842b02b0 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:17.392977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d73b5-30c6-4f63-b7a7-0b8cda1926fa None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.393513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d73b5-30c6-4f63-b7a7-0b8cda1926fa None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.393797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d73b5-30c6-4f63-b7a7-0b8cda1926fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.410907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-1c8d73b5-30c6-4f63-b7a7-0b8cda1926fa None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:17.412106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1c8d73b5-30c6-4f63-b7a7-0b8cda1926fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:17.474173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1214/2493] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:17.484531 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:17.486094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:17.548529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb846a-433b-4e67-8cdc-3bb5dfacdfa6 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:17.552352 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137dae4-377e-4c2d-80e6-6f674c34a1ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:17.553039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137dae4-377e-4c2d-80e6-6f674c34a1ab service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.553325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137dae4-377e-4c2d-80e6-6f674c34a1ab service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.553636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137dae4-377e-4c2d-80e6-6f674c34a1ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.563416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137dae4-377e-4c2d-80e6-6f674c34a1ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:17.564853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137dae4-377e-4c2d-80e6-6f674c34a1ab service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:17.574874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1280/2494] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:17.870010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d73b5-30c6-4f63-b7a7-0b8cda1926fa None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:17.871519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1c8d73b5-30c6-4f63-b7a7-0b8cda1926fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:17.886871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d73b5-30c6-4f63-b7a7-0b8cda1926fa None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:17.893332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:17.894320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.894680 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.895167 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] PATH_INFO: `/v3/projects/bd6b521e4fa241a984857a94fbde044e` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.896834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd6b521e4fa241a984857a94fbde044e)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:17.899802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:17.976583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd6b521e4fa241a984857a94fbde044e', 'request_id': 'req-94e65e91-dd1b-490f-9373-6dfa95841053'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.983571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd6b521e4fa241a984857a94fbde044e', 'request_id': 'req-94e65e91-dd1b-490f-9373-6dfa95841053'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.990071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd6b521e4fa241a984857a94fbde044e', 'request_id': 'req-94e65e91-dd1b-490f-9373-6dfa95841053'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:18.003749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] The token cache is being invalidate because project bd6b521e4fa241a984857a94fbde044e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:59:18.004048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:18.004696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-94e65e91-dd1b-490f-9373-6dfa95841053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:18.006934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1215/2495] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:59:17 2022] DELETE /identity/v3/projects/bd6b521e4fa241a984857a94fbde044e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:59:18.011850 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f4382-6ade-428a-beb1-bbb73c2eda4b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.012153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f4382-6ade-428a-beb1-bbb73c2eda4b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.012925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f4382-6ade-428a-beb1-bbb73c2eda4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.032632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-481f4382-6ade-428a-beb1-bbb73c2eda4b None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:18.033922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-481f4382-6ade-428a-beb1-bbb73c2eda4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:18.091274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:18.107902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1281/2496] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:18.117726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753ee38-6aac-4fe4-ac3b-0040a9ee7d62 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.118123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753ee38-6aac-4fe4-ac3b-0040a9ee7d62 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.118472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753ee38-6aac-4fe4-ac3b-0040a9ee7d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.147370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a753ee38-6aac-4fe4-ac3b-0040a9ee7d62 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:18.149589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a753ee38-6aac-4fe4-ac3b-0040a9ee7d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:18.160538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1216/2497] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:18.164953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f4382-6ade-428a-beb1-bbb73c2eda4b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.175108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f4382-6ade-428a-beb1-bbb73c2eda4b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:18.180864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8578a05-297d-41e7-bf9e-9dabadef7422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:18.184155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8578a05-297d-41e7-bf9e-9dabadef7422 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.184751 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8578a05-297d-41e7-bf9e-9dabadef7422 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.185012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8578a05-297d-41e7-bf9e-9dabadef7422 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.185899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8578a05-297d-41e7-bf9e-9dabadef7422 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:59:18.186212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8578a05-297d-41e7-bf9e-9dabadef7422 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:18.187914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8578a05-297d-41e7-bf9e-9dabadef7422 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:18.205536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1282/2498] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:59:18.207061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753ee38-6aac-4fe4-ac3b-0040a9ee7d62 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.212824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:18.217958 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753ee38-6aac-4fe4-ac3b-0040a9ee7d62 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:18.224227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173d3f3-9622-4ca9-be7f-3e41b4b0153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:18.225532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173d3f3-9622-4ca9-be7f-3e41b4b0153c admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.225665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173d3f3-9622-4ca9-be7f-3e41b4b0153c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.225997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173d3f3-9622-4ca9-be7f-3e41b4b0153c admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.226717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173d3f3-9622-4ca9-be7f-3e41b4b0153c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:18.228700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173d3f3-9622-4ca9-be7f-3e41b4b0153c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:18.269502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1217/2499] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:59:18 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:59:18.275436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8578a05-297d-41e7-bf9e-9dabadef7422 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.286360 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8578a05-297d-41e7-bf9e-9dabadef7422 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:18.369511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0706998-d1cf-48c9-8d6e-c504a7dafbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:18.370203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0706998-d1cf-48c9-8d6e-c504a7dafbd2 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.370489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0706998-d1cf-48c9-8d6e-c504a7dafbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.370770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0706998-d1cf-48c9-8d6e-c504a7dafbd2 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.371491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0706998-d1cf-48c9-8d6e-c504a7dafbd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:18.374172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0706998-d1cf-48c9-8d6e-c504a7dafbd2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:18.399305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1283/2500] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:59:18 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:18.403978 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173d3f3-9622-4ca9-be7f-3e41b4b0153c admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.416819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173d3f3-9622-4ca9-be7f-3e41b4b0153c admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:18.421780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b94d2c-2efd-4091-a878-539f22e03038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:18.422468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94d2c-2efd-4091-a878-539f22e03038 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.422751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94d2c-2efd-4091-a878-539f22e03038 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.423037 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94d2c-2efd-4091-a878-539f22e03038 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.423774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b94d2c-2efd-4091-a878-539f22e03038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:18.425418 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b94d2c-2efd-4091-a878-539f22e03038 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:18.437833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1218/2501] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:18.441958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0706998-d1cf-48c9-8d6e-c504a7dafbd2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.453643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0706998-d1cf-48c9-8d6e-c504a7dafbd2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:18.458795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e335e-68fb-42e3-81ce-9a720555db79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:18.459656 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e335e-68fb-42e3-81ce-9a720555db79 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.459830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e335e-68fb-42e3-81ce-9a720555db79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.460111 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e335e-68fb-42e3-81ce-9a720555db79 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.460858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e335e-68fb-42e3-81ce-9a720555db79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:18.462494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e335e-68fb-42e3-81ce-9a720555db79 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:18.473302 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1284/2502] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:18.478889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b94d2c-2efd-4091-a878-539f22e03038 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.489381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b94d2c-2efd-4091-a878-539f22e03038 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:18.494589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62fe8e-c108-41c8-9b3a-3ed2e64a524e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:18.495489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fe8e-c108-41c8-9b3a-3ed2e64a524e admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.495766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fe8e-c108-41c8-9b3a-3ed2e64a524e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.496041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fe8e-c108-41c8-9b3a-3ed2e64a524e admin admin] PATH_INFO: `/v3/projects/14ca5d3c21cd4a309b4f48f947bebf6b/users/39d780775b2f4f9098d8655dfa233a50/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.514459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62fe8e-c108-41c8-9b3a-3ed2e64a524e admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ca5d3c21cd4a309b4f48f947bebf6b, user_id=39d780775b2f4f9098d8655dfa233a50, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:18.517136 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62fe8e-c108-41c8-9b3a-3ed2e64a524e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:18.531530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1219/2503] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:18 2022] PUT /identity/v3/projects/14ca5d3c21cd4a309b4f48f947bebf6b/users/39d780775b2f4f9098d8655dfa233a50/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:18.542333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529e6b4-437a-44a9-b9fe-c15ea3af9c0e None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.542625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529e6b4-437a-44a9-b9fe-c15ea3af9c0e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.542945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529e6b4-437a-44a9-b9fe-c15ea3af9c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.571184 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-3529e6b4-437a-44a9-b9fe-c15ea3af9c0e None None] MFA Rules not processed for user `39d780775b2f4f9098d8655dfa233a50`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:18.572434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3529e6b4-437a-44a9-b9fe-c15ea3af9c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:18.641748 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1285/2504] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:18.647182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62fe8e-c108-41c8-9b3a-3ed2e64a524e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.714688 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62fe8e-c108-41c8-9b3a-3ed2e64a524e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:18.719878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62faf254-e8e4-465c-91e5-1b213421c17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:18.720547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62faf254-e8e4-465c-91e5-1b213421c17d admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.720834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62faf254-e8e4-465c-91e5-1b213421c17d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.721136 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62faf254-e8e4-465c-91e5-1b213421c17d admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.721875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62faf254-e8e4-465c-91e5-1b213421c17d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:18.724055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62faf254-e8e4-465c-91e5-1b213421c17d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:18.752799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1220/2505] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:59:18 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:18.757219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529e6b4-437a-44a9-b9fe-c15ea3af9c0e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.768667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529e6b4-437a-44a9-b9fe-c15ea3af9c0e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:18.818780 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:18.829336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8d9c2-01e2-4f9c-8727-eace2ef25845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:18.830039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8d9c2-01e2-4f9c-8727-eace2ef25845 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.830329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8d9c2-01e2-4f9c-8727-eace2ef25845 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.830613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8d9c2-01e2-4f9c-8727-eace2ef25845 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.831312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8d9c2-01e2-4f9c-8727-eace2ef25845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:18.833228 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8d9c2-01e2-4f9c-8727-eace2ef25845 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:18.860451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1286/2506] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:59:18 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:59:18.865675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62faf254-e8e4-465c-91e5-1b213421c17d admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.878542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62faf254-e8e4-465c-91e5-1b213421c17d admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:18.883638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2da7ac-ed1c-4993-88d9-ef455a87b013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:18.884366 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2da7ac-ed1c-4993-88d9-ef455a87b013 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.884654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2da7ac-ed1c-4993-88d9-ef455a87b013 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.884935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2da7ac-ed1c-4993-88d9-ef455a87b013 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.885720 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2da7ac-ed1c-4993-88d9-ef455a87b013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:18.887836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2da7ac-ed1c-4993-88d9-ef455a87b013 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:18.899665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1221/2507] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:18.904566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8d9c2-01e2-4f9c-8727-eace2ef25845 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.915321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8d9c2-01e2-4f9c-8727-eace2ef25845 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:18.920314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72f2c3-f4c1-4bce-b738-4519216246e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:18.921198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72f2c3-f4c1-4bce-b738-4519216246e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.921519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72f2c3-f4c1-4bce-b738-4519216246e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.921832 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72f2c3-f4c1-4bce-b738-4519216246e1 admin admin] PATH_INFO: `/v3/projects/5d115fcd8a4640449fa1160138057570/users/67574db120b74be1a120bc65dbf603f8/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.938650 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72f2c3-f4c1-4bce-b738-4519216246e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d115fcd8a4640449fa1160138057570, user_id=67574db120b74be1a120bc65dbf603f8, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:18.941461 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72f2c3-f4c1-4bce-b738-4519216246e1 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:18.958782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1287/2508] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:18 2022] PUT /identity/v3/projects/5d115fcd8a4640449fa1160138057570/users/67574db120b74be1a120bc65dbf603f8/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:18.963943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2da7ac-ed1c-4993-88d9-ef455a87b013 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.975013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2da7ac-ed1c-4993-88d9-ef455a87b013 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:19.033181 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da545ec0-52c4-441d-9cb2-77b4eb3073d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:19.033887 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da545ec0-52c4-441d-9cb2-77b4eb3073d9 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:19.034178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da545ec0-52c4-441d-9cb2-77b4eb3073d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:19.034461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da545ec0-52c4-441d-9cb2-77b4eb3073d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:19.035197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da545ec0-52c4-441d-9cb2-77b4eb3073d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:19.036813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da545ec0-52c4-441d-9cb2-77b4eb3073d9 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:19.047412 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1222/2509] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:19.052095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72f2c3-f4c1-4bce-b738-4519216246e1 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:19.063405 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72f2c3-f4c1-4bce-b738-4519216246e1 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:19.068209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679ac51-3442-4b98-b017-63c4876943c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:19.069133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679ac51-3442-4b98-b017-63c4876943c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:19.069451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679ac51-3442-4b98-b017-63c4876943c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:19.069732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679ac51-3442-4b98-b017-63c4876943c5 admin admin] PATH_INFO: `/v3/projects/5d115fcd8a4640449fa1160138057570/users/67574db120b74be1a120bc65dbf603f8/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:19.071736 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679ac51-3442-4b98-b017-63c4876943c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d115fcd8a4640449fa1160138057570, user_id=67574db120b74be1a120bc65dbf603f8, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:19.074416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679ac51-3442-4b98-b017-63c4876943c5 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:19.086806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1288/2510] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:19 2022] PUT /identity/v3/projects/5d115fcd8a4640449fa1160138057570/users/67574db120b74be1a120bc65dbf603f8/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:19.099631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0e2e0-8a57-4686-adff-f9268dfdb993 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:19.099919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0e2e0-8a57-4686-adff-f9268dfdb993 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:19.100260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0e2e0-8a57-4686-adff-f9268dfdb993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:19.128447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-8ca0e2e0-8a57-4686-adff-f9268dfdb993 None None] MFA Rules not processed for user `67574db120b74be1a120bc65dbf603f8`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:19.129737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8ca0e2e0-8a57-4686-adff-f9268dfdb993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:19.134867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:19.207923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1223/2511] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:19.220042 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679ac51-3442-4b98-b017-63c4876943c5 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:19.222372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b679ac51-3442-4b98-b017-63c4876943c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:19.281663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679ac51-3442-4b98-b017-63c4876943c5 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:19.285171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba8873-baee-4781-8416-69d7cb3c1dca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:19.285847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba8873-baee-4781-8416-69d7cb3c1dca service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:19.286132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba8873-baee-4781-8416-69d7cb3c1dca service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:19.286411 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba8873-baee-4781-8416-69d7cb3c1dca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:19.295905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba8873-baee-4781-8416-69d7cb3c1dca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:19.297325 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba8873-baee-4781-8416-69d7cb3c1dca service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:19.307106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1289/2512] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:19.397321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0e2e0-8a57-4686-adff-f9268dfdb993 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:19.407429 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0e2e0-8a57-4686-adff-f9268dfdb993 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:19.411508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df1ada-68b4-4b26-973f-65e70696ce14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:19.412162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df1ada-68b4-4b26-973f-65e70696ce14 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:19.412447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df1ada-68b4-4b26-973f-65e70696ce14 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:19.412726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df1ada-68b4-4b26-973f-65e70696ce14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:19.475459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df1ada-68b4-4b26-973f-65e70696ce14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:19.477326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df1ada-68b4-4b26-973f-65e70696ce14 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:19.488757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1224/2513] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:29.302666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=47,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:29.484046 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=54 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:30.669704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba8873-baee-4781-8416-69d7cb3c1dca service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:30.671108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-71ba8873-baee-4781-8416-69d7cb3c1dca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:30.768365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba8873-baee-4781-8416-69d7cb3c1dca service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:30.774420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958288c2-31ea-41a4-b576-4dac4b286fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:30.775906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958288c2-31ea-41a4-b576-4dac4b286fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:30.776436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958288c2-31ea-41a4-b576-4dac4b286fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:30.776948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958288c2-31ea-41a4-b576-4dac4b286fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:30.780866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-958288c2-31ea-41a4-b576-4dac4b286fa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:30.894731 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958288c2-31ea-41a4-b576-4dac4b286fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:30.896436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958288c2-31ea-41a4-b576-4dac4b286fa9 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:30.910662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1290/2514] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:34.844534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df1ada-68b4-4b26-973f-65e70696ce14 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:34.844534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e8df1ada-68b4-4b26-973f-65e70696ce14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:34.871237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df1ada-68b4-4b26-973f-65e70696ce14 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:34.879895 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa77195-ba40-4114-ba22-d8e466fc50f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:34.881789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa77195-ba40-4114-ba22-d8e466fc50f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:34.882505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa77195-ba40-4114-ba22-d8e466fc50f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:34.883341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa77195-ba40-4114-ba22-d8e466fc50f8 admin admin] PATH_INFO: `/v3/users/af989ff6dc024ff19023e6f633915ec3` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:34.907428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa77195-ba40-4114-ba22-d8e466fc50f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af989ff6dc024ff19023e6f633915ec3)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:34.909690 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa77195-ba40-4114-ba22-d8e466fc50f8 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:34.914936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958288c2-31ea-41a4-b576-4dac4b286fa9 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:34.928961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958288c2-31ea-41a4-b576-4dac4b286fa9 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:34.945402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331861e1-f94f-45f1-95b9-328b3a353274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:34.947273 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331861e1-f94f-45f1-95b9-328b3a353274 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:34.948037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331861e1-f94f-45f1-95b9-328b3a353274 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:34.948616 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331861e1-f94f-45f1-95b9-328b3a353274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:34.952192 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-331861e1-f94f-45f1-95b9-328b3a353274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:34.972878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331861e1-f94f-45f1-95b9-328b3a353274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:34.975786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331861e1-f94f-45f1-95b9-328b3a353274 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:34.979463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-aaa77195-ba40-4114-ba22-d8e466fc50f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af989ff6dc024ff19023e6f633915ec3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:34.989320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-aaa77195-ba40-4114-ba22-d8e466fc50f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af989ff6dc024ff19023e6f633915ec3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:34.991306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1291/2515] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:59:35.001726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-aaa77195-ba40-4114-ba22-d8e466fc50f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af989ff6dc024ff19023e6f633915ec3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:35.015569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-aaa77195-ba40-4114-ba22-d8e466fc50f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af989ff6dc024ff19023e6f633915ec3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:35.016381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-aaa77195-ba40-4114-ba22-d8e466fc50f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af989ff6dc024ff19023e6f633915ec3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:35.023065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1225/2516] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:59:34 2022] DELETE /identity/v3/users/af989ff6dc024ff19023e6f633915ec3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:35.231346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331861e1-f94f-45f1-95b9-328b3a353274 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:35.247978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331861e1-f94f-45f1-95b9-328b3a353274 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:35.356447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:35.357778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:35.358296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:35.358827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] PATH_INFO: `/v3/projects/5da6c42976154440bd0459beb12477f2` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:35.382124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5da6c42976154440bd0459beb12477f2)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:35.385281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:35.425323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:35.448840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5da6c42976154440bd0459beb12477f2', 'request_id': 'req-20bf7aa5-1f30-480b-8b15-839121571631'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:35.459960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5da6c42976154440bd0459beb12477f2', 'request_id': 'req-20bf7aa5-1f30-480b-8b15-839121571631'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:35.481419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5da6c42976154440bd0459beb12477f2', 'request_id': 'req-20bf7aa5-1f30-480b-8b15-839121571631'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:35.496391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] The token cache is being invalidate because project 5da6c42976154440bd0459beb12477f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:59:35.496936 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:35.503617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-20bf7aa5-1f30-480b-8b15-839121571631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:35.510903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1292/2517] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:59:35 2022] DELETE /identity/v3/projects/5da6c42976154440bd0459beb12477f2 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:35.518222 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd0cf4-8995-4c11-89fb-c8c546f1c7b9 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:35.519008 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd0cf4-8995-4c11-89fb-c8c546f1c7b9 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:35.519648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd0cf4-8995-4c11-89fb-c8c546f1c7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:35.557832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-f5bd0cf4-8995-4c11-89fb-c8c546f1c7b9 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:35.561067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-f5bd0cf4-8995-4c11-89fb-c8c546f1c7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:35.690937 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1226/2518] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:35.700787 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e9037-38f0-4a89-badc-0402117858c4 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:35.701922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e9037-38f0-4a89-badc-0402117858c4 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:35.702189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e9037-38f0-4a89-badc-0402117858c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:35.727597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-318e9037-38f0-4a89-badc-0402117858c4 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:35.729077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-318e9037-38f0-4a89-badc-0402117858c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:35.741434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1293/2519] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:35.743528 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd0cf4-8995-4c11-89fb-c8c546f1c7b9 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:35.754413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd0cf4-8995-4c11-89fb-c8c546f1c7b9 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:35.759726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd57fa3-22ab-4c1e-8cfe-5378ea368875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:35.759986 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57fa3-22ab-4c1e-8cfe-5378ea368875 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:35.760250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57fa3-22ab-4c1e-8cfe-5378ea368875 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:35.760546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57fa3-22ab-4c1e-8cfe-5378ea368875 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:35.761257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd57fa3-22ab-4c1e-8cfe-5378ea368875 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:59:35.761599 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd57fa3-22ab-4c1e-8cfe-5378ea368875 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:35.764010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd57fa3-22ab-4c1e-8cfe-5378ea368875 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:35.779161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1227/2520] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:59:35.783947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e9037-38f0-4a89-badc-0402117858c4 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:35.793793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e9037-38f0-4a89-badc-0402117858c4 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:35.798549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81932dd-9d61-4475-a3bc-09834d6c4437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:35.799217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81932dd-9d61-4475-a3bc-09834d6c4437 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:35.799651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81932dd-9d61-4475-a3bc-09834d6c4437 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:35.799937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81932dd-9d61-4475-a3bc-09834d6c4437 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:35.800627 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81932dd-9d61-4475-a3bc-09834d6c4437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:35.802572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81932dd-9d61-4475-a3bc-09834d6c4437 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:35.837394 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1294/2521] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:59:35 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:35.841609 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd57fa3-22ab-4c1e-8cfe-5378ea368875 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:35.855244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd57fa3-22ab-4c1e-8cfe-5378ea368875 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:35.909520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:35.928738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09949e-ea5f-47ee-8776-176b66f9d685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:35.929940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09949e-ea5f-47ee-8776-176b66f9d685 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:35.930418 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09949e-ea5f-47ee-8776-176b66f9d685 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:35.930924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09949e-ea5f-47ee-8776-176b66f9d685 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:35.932273 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09949e-ea5f-47ee-8776-176b66f9d685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:35.936806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09949e-ea5f-47ee-8776-176b66f9d685 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:35.977075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1228/2522] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:59:35 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:59:35.985334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81932dd-9d61-4475-a3bc-09834d6c4437 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:35.999463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81932dd-9d61-4475-a3bc-09834d6c4437 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:36.005684 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9aa8d-5566-497b-895d-ccca28a4fca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:36.006377 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9aa8d-5566-497b-895d-ccca28a4fca9 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:36.006664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9aa8d-5566-497b-895d-ccca28a4fca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:36.006941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9aa8d-5566-497b-895d-ccca28a4fca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:36.007844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9aa8d-5566-497b-895d-ccca28a4fca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:36.009657 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9aa8d-5566-497b-895d-ccca28a4fca9 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:36.021525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1295/2523] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:36.027308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09949e-ea5f-47ee-8776-176b66f9d685 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:36.043608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09949e-ea5f-47ee-8776-176b66f9d685 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:36.048559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ae3ef-5eab-4ca9-9a05-9549cc9ac0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:36.049264 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ae3ef-5eab-4ca9-9a05-9549cc9ac0bd admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:36.049583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ae3ef-5eab-4ca9-9a05-9549cc9ac0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:36.050007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ae3ef-5eab-4ca9-9a05-9549cc9ac0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:36.051027 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ae3ef-5eab-4ca9-9a05-9549cc9ac0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:36.054101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ae3ef-5eab-4ca9-9a05-9549cc9ac0bd admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:36.078783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1229/2524] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:36.080455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9aa8d-5566-497b-895d-ccca28a4fca9 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:36.093231 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9aa8d-5566-497b-895d-ccca28a4fca9 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:36.099690 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd8303-95e1-482c-9834-d79ef0ee07fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:36.100900 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd8303-95e1-482c-9834-d79ef0ee07fe admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:36.101107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd8303-95e1-482c-9834-d79ef0ee07fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:36.101404 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd8303-95e1-482c-9834-d79ef0ee07fe admin admin] PATH_INFO: `/v3/projects/1100541a02904558b72f70d1483d9425/users/e7f8113c231d4c2499630b2f6c83a07a/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:36.129339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd8303-95e1-482c-9834-d79ef0ee07fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1100541a02904558b72f70d1483d9425, user_id=e7f8113c231d4c2499630b2f6c83a07a, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:36.135349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd8303-95e1-482c-9834-d79ef0ee07fe admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:36.159081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1296/2525] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:36 2022] PUT /identity/v3/projects/1100541a02904558b72f70d1483d9425/users/e7f8113c231d4c2499630b2f6c83a07a/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:59:36.175143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eb001-5dae-4339-be54-324937f3f4d1 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:36.175442 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eb001-5dae-4339-be54-324937f3f4d1 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:36.175707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eb001-5dae-4339-be54-324937f3f4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:36.224486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-8e7eb001-5dae-4339-be54-324937f3f4d1 None None] MFA Rules not processed for user `e7f8113c231d4c2499630b2f6c83a07a`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:36.226019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8e7eb001-5dae-4339-be54-324937f3f4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:36.376644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1230/2526] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:59:36.379468 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd8303-95e1-482c-9834-d79ef0ee07fe admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:36.500926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd8303-95e1-482c-9834-d79ef0ee07fe admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:36.506140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040e729-d6d2-4dc0-b3cc-68bfdfefe62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:36.506961 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040e729-d6d2-4dc0-b3cc-68bfdfefe62d admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:36.507285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040e729-d6d2-4dc0-b3cc-68bfdfefe62d admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:36.507828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040e729-d6d2-4dc0-b3cc-68bfdfefe62d admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:36.508292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040e729-d6d2-4dc0-b3cc-68bfdfefe62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:36.510552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040e729-d6d2-4dc0-b3cc-68bfdfefe62d admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:36.562458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1297/2527] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:59:36 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:36.568520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eb001-5dae-4339-be54-324937f3f4d1 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:36.591178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eb001-5dae-4339-be54-324937f3f4d1 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:36.701327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701116b-f8cc-4cdf-a0bb-56efc69832a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:36.702538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701116b-f8cc-4cdf-a0bb-56efc69832a4 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:36.703032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701116b-f8cc-4cdf-a0bb-56efc69832a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:36.703503 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701116b-f8cc-4cdf-a0bb-56efc69832a4 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:36.704833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701116b-f8cc-4cdf-a0bb-56efc69832a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:36.708627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701116b-f8cc-4cdf-a0bb-56efc69832a4 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:36.749413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1231/2528] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:59:36 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:59:36.755164 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040e729-d6d2-4dc0-b3cc-68bfdfefe62d admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:36.772058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040e729-d6d2-4dc0-b3cc-68bfdfefe62d admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:36.783772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67adbd4-f02d-438e-bf4b-d4af21c9c0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:36.784847 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67adbd4-f02d-438e-bf4b-d4af21c9c0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:36.785145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67adbd4-f02d-438e-bf4b-d4af21c9c0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:36.785472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67adbd4-f02d-438e-bf4b-d4af21c9c0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:36.786295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67adbd4-f02d-438e-bf4b-d4af21c9c0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:36.789692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67adbd4-f02d-438e-bf4b-d4af21c9c0c2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:36.803591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1298/2529] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:36.810141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701116b-f8cc-4cdf-a0bb-56efc69832a4 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:36.829910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701116b-f8cc-4cdf-a0bb-56efc69832a4 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:36.838369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee770a71-89ea-4334-8e5a-6a3b068d8484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:36.840612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee770a71-89ea-4334-8e5a-6a3b068d8484 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:36.840960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee770a71-89ea-4334-8e5a-6a3b068d8484 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:36.841344 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee770a71-89ea-4334-8e5a-6a3b068d8484 admin admin] PATH_INFO: `/v3/projects/9791dde84a804e73ac854a0339da3600/users/6a144ad0d31748719d03311406e02cb0/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:36.886277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee770a71-89ea-4334-8e5a-6a3b068d8484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9791dde84a804e73ac854a0339da3600, user_id=6a144ad0d31748719d03311406e02cb0, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:36.892338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee770a71-89ea-4334-8e5a-6a3b068d8484 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:36.911901 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1232/2530] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:36 2022] PUT /identity/v3/projects/9791dde84a804e73ac854a0339da3600/users/6a144ad0d31748719d03311406e02cb0/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:36.915933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67adbd4-f02d-438e-bf4b-d4af21c9c0c2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:36.927209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67adbd4-f02d-438e-bf4b-d4af21c9c0c2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:37.001308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:37.073111 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d404ebc-9f7f-4fa9-a087-64cd7f28b39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:37.073835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d404ebc-9f7f-4fa9-a087-64cd7f28b39b admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:37.074122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d404ebc-9f7f-4fa9-a087-64cd7f28b39b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:37.074475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d404ebc-9f7f-4fa9-a087-64cd7f28b39b admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:37.075765 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d404ebc-9f7f-4fa9-a087-64cd7f28b39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:37.077503 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d404ebc-9f7f-4fa9-a087-64cd7f28b39b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:37.094221 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1299/2531] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:37.099377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee770a71-89ea-4334-8e5a-6a3b068d8484 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:37.120186 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee770a71-89ea-4334-8e5a-6a3b068d8484 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:37.129822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f08b21-b5a9-44bc-a322-abb45ff33bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:37.131406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f08b21-b5a9-44bc-a322-abb45ff33bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:37.132826 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f08b21-b5a9-44bc-a322-abb45ff33bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:37.133436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f08b21-b5a9-44bc-a322-abb45ff33bc6 admin admin] PATH_INFO: `/v3/projects/9791dde84a804e73ac854a0339da3600/users/6a144ad0d31748719d03311406e02cb0/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:37.138190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f08b21-b5a9-44bc-a322-abb45ff33bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9791dde84a804e73ac854a0339da3600, user_id=6a144ad0d31748719d03311406e02cb0, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:37.142876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f08b21-b5a9-44bc-a322-abb45ff33bc6 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:37.185906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1233/2532] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:37 2022] PUT /identity/v3/projects/9791dde84a804e73ac854a0339da3600/users/6a144ad0d31748719d03311406e02cb0/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:59:37.187516 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956b89-4bd6-4bf0-bc32-b52c6ea7686b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:37.188396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956b89-4bd6-4bf0-bc32-b52c6ea7686b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:37.188396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5956b89-4bd6-4bf0-bc32-b52c6ea7686b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:37.282420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-b5956b89-4bd6-4bf0-bc32-b52c6ea7686b None None] MFA Rules not processed for user `6a144ad0d31748719d03311406e02cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:37.284128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b5956b89-4bd6-4bf0-bc32-b52c6ea7686b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:37.419666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1300/2533] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:37.430903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f08b21-b5a9-44bc-a322-abb45ff33bc6 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:37.434924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-84f08b21-b5a9-44bc-a322-abb45ff33bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:37.455805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:37.560375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f08b21-b5a9-44bc-a322-abb45ff33bc6 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:37.565797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98bda1-0b7c-4667-892a-838c79526087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:37.566479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98bda1-0b7c-4667-892a-838c79526087 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:37.566813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98bda1-0b7c-4667-892a-838c79526087 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:37.567242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98bda1-0b7c-4667-892a-838c79526087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:37.649108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98bda1-0b7c-4667-892a-838c79526087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:37.650603 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98bda1-0b7c-4667-892a-838c79526087 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:37.664030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1234/2534] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:39.290767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5956b89-4bd6-4bf0-bc32-b52c6ea7686b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:39.294208 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b5956b89-4bd6-4bf0-bc32-b52c6ea7686b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:39.391223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5956b89-4bd6-4bf0-bc32-b52c6ea7686b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:39.396545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af8a44-eb95-4386-a3b6-4b0dec6ea92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:39.397581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af8a44-eb95-4386-a3b6-4b0dec6ea92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:39.397926 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af8a44-eb95-4386-a3b6-4b0dec6ea92b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:39.398434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af8a44-eb95-4386-a3b6-4b0dec6ea92b admin admin] PATH_INFO: `/v3/users/31790576f8734f75adbbd8e9c0d61721` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:39.415730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af8a44-eb95-4386-a3b6-4b0dec6ea92b admin admin] RBAC: Authorizing `identity:delete_user(user_id=31790576f8734f75adbbd8e9c0d61721)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:39.417566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af8a44-eb95-4386-a3b6-4b0dec6ea92b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:39.473689 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:39.482143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-91af8a44-eb95-4386-a3b6-4b0dec6ea92b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31790576f8734f75adbbd8e9c0d61721'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:39.494987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-91af8a44-eb95-4386-a3b6-4b0dec6ea92b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31790576f8734f75adbbd8e9c0d61721'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:39.507612 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-91af8a44-eb95-4386-a3b6-4b0dec6ea92b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31790576f8734f75adbbd8e9c0d61721'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:39.508257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-91af8a44-eb95-4386-a3b6-4b0dec6ea92b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31790576f8734f75adbbd8e9c0d61721'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:39.517233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-91af8a44-eb95-4386-a3b6-4b0dec6ea92b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31790576f8734f75adbbd8e9c0d61721'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:39.524626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1301/2535] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:59:39 2022] DELETE /identity/v3/users/31790576f8734f75adbbd8e9c0d61721 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:39.732654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98bda1-0b7c-4667-892a-838c79526087 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:39.755646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98bda1-0b7c-4667-892a-838c79526087 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:39.835952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:39.836832 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:39.837110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:39.837415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] PATH_INFO: `/v3/projects/8298856e2d28408296cc7e4deca84846` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:39.854358 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=8298856e2d28408296cc7e4deca84846)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:39.856854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:39.904920 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8298856e2d28408296cc7e4deca84846', 'request_id': 'req-7e354585-ef87-4b31-af72-38969014a0ee'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:39.911378 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8298856e2d28408296cc7e4deca84846', 'request_id': 'req-7e354585-ef87-4b31-af72-38969014a0ee'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:39.913325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:39.918403 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8298856e2d28408296cc7e4deca84846', 'request_id': 'req-7e354585-ef87-4b31-af72-38969014a0ee'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:39.931737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] The token cache is being invalidate because project 8298856e2d28408296cc7e4deca84846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:59:39.932023 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:39.932639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7e354585-ef87-4b31-af72-38969014a0ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:39.934908 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1235/2536] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:59:39 2022] DELETE /identity/v3/projects/8298856e2d28408296cc7e4deca84846 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:39.939895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d116e-4dee-4aa4-bfb6-c604e3c8690a None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:39.940191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d116e-4dee-4aa4-bfb6-c604e3c8690a None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:39.940480 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d116e-4dee-4aa4-bfb6-c604e3c8690a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:39.958574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-158d116e-4dee-4aa4-bfb6-c604e3c8690a None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:39.959815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-158d116e-4dee-4aa4-bfb6-c604e3c8690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:40.022312 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1302/2537] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:40.030833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c7a0a-a05a-4c14-9e4b-35d36d99d348 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:40.031129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c7a0a-a05a-4c14-9e4b-35d36d99d348 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:40.031411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c7a0a-a05a-4c14-9e4b-35d36d99d348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:40.050298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-5e4c7a0a-a05a-4c14-9e4b-35d36d99d348 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:40.051551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5e4c7a0a-a05a-4c14-9e4b-35d36d99d348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:40.059691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1236/2538] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:40.064096 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d116e-4dee-4aa4-bfb6-c604e3c8690a None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:40.073901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d116e-4dee-4aa4-bfb6-c604e3c8690a None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:40.078499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017440e2-338d-4969-bda6-dd5cd79f36ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:40.079176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017440e2-338d-4969-bda6-dd5cd79f36ab admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:40.079458 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017440e2-338d-4969-bda6-dd5cd79f36ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:40.079734 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017440e2-338d-4969-bda6-dd5cd79f36ab admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:40.080467 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017440e2-338d-4969-bda6-dd5cd79f36ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:59:40.080771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017440e2-338d-4969-bda6-dd5cd79f36ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:40.082385 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017440e2-338d-4969-bda6-dd5cd79f36ab admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:40.094721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1303/2539] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:59:40.099272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c7a0a-a05a-4c14-9e4b-35d36d99d348 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:40.109007 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c7a0a-a05a-4c14-9e4b-35d36d99d348 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:40.113688 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037b386-8274-48d4-a703-11c940c118a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:40.114335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037b386-8274-48d4-a703-11c940c118a8 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:40.114615 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037b386-8274-48d4-a703-11c940c118a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:40.114889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037b386-8274-48d4-a703-11c940c118a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:40.115585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037b386-8274-48d4-a703-11c940c118a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:40.117483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037b386-8274-48d4-a703-11c940c118a8 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:40.143179 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1237/2540] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:59:40 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:40.147528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017440e2-338d-4969-bda6-dd5cd79f36ab admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:40.158200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017440e2-338d-4969-bda6-dd5cd79f36ab admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:40.218981 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da867e-a7d7-43d3-b65b-7288b94fc4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:40.219683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da867e-a7d7-43d3-b65b-7288b94fc4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:40.219963 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da867e-a7d7-43d3-b65b-7288b94fc4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:40.220241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da867e-a7d7-43d3-b65b-7288b94fc4c2 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:40.220935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da867e-a7d7-43d3-b65b-7288b94fc4c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:40.223175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da867e-a7d7-43d3-b65b-7288b94fc4c2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:40.246373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1304/2541] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:59:40 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:59:40.250843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037b386-8274-48d4-a703-11c940c118a8 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:40.261518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037b386-8274-48d4-a703-11c940c118a8 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:40.266306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe878a4-18bf-46e4-b152-dd8621dcb488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:40.267013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe878a4-18bf-46e4-b152-dd8621dcb488 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:40.267292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe878a4-18bf-46e4-b152-dd8621dcb488 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:40.267566 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe878a4-18bf-46e4-b152-dd8621dcb488 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:40.268310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe878a4-18bf-46e4-b152-dd8621dcb488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:40.269979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe878a4-18bf-46e4-b152-dd8621dcb488 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:40.281749 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1238/2542] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:40.286081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da867e-a7d7-43d3-b65b-7288b94fc4c2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:40.297490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da867e-a7d7-43d3-b65b-7288b94fc4c2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:40.302567 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee0ec4-9f39-4749-8ccf-ebe05449e82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:40.303246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee0ec4-9f39-4749-8ccf-ebe05449e82b admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:40.303585 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee0ec4-9f39-4749-8ccf-ebe05449e82b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:40.303866 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee0ec4-9f39-4749-8ccf-ebe05449e82b admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:40.304666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee0ec4-9f39-4749-8ccf-ebe05449e82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:40.306328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee0ec4-9f39-4749-8ccf-ebe05449e82b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:40.318714 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1305/2543] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:40.323101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe878a4-18bf-46e4-b152-dd8621dcb488 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:40.335214 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe878a4-18bf-46e4-b152-dd8621dcb488 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:40.341320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f0908-d5cf-427e-8f88-183631f58e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:40.342259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f0908-d5cf-427e-8f88-183631f58e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:40.342543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f0908-d5cf-427e-8f88-183631f58e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:40.342819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f0908-d5cf-427e-8f88-183631f58e3f admin admin] PATH_INFO: `/v3/projects/fc46a5fc6c9240ad9234a97d66627fb9/users/f872791891f44303801e441459245005/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:40.360471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f0908-d5cf-427e-8f88-183631f58e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc46a5fc6c9240ad9234a97d66627fb9, user_id=f872791891f44303801e441459245005, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:40.363234 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f0908-d5cf-427e-8f88-183631f58e3f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:40.380913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1239/2544] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:40 2022] PUT /identity/v3/projects/fc46a5fc6c9240ad9234a97d66627fb9/users/f872791891f44303801e441459245005/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:40.392537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00e1e6-d56c-445c-a770-45fc11db9faf None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:40.392825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00e1e6-d56c-445c-a770-45fc11db9faf None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:40.393100 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00e1e6-d56c-445c-a770-45fc11db9faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:40.421438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-be00e1e6-d56c-445c-a770-45fc11db9faf None None] MFA Rules not processed for user `f872791891f44303801e441459245005`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:40.422165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-be00e1e6-d56c-445c-a770-45fc11db9faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:40.435619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:40.491440 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1306/2545] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:40.502932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f0908-d5cf-427e-8f88-183631f58e3f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:40.504650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e02f0908-d5cf-427e-8f88-183631f58e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:40.579970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f0908-d5cf-427e-8f88-183631f58e3f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:40.585614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7c373-5152-4f20-b16e-467baa990529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:40.586265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7c373-5152-4f20-b16e-467baa990529 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:40.586553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7c373-5152-4f20-b16e-467baa990529 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:40.586830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7c373-5152-4f20-b16e-467baa990529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:40.595857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7c373-5152-4f20-b16e-467baa990529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:40.597468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7c373-5152-4f20-b16e-467baa990529 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:40.606705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1240/2546] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:40.837224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00e1e6-d56c-445c-a770-45fc11db9faf None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:40.911987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00e1e6-d56c-445c-a770-45fc11db9faf None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:40.917452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:40.918513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:40.918796 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:40.919242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] PATH_INFO: `/v3/users/f872791891f44303801e441459245005` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:40.920487 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f872791891f44303801e441459245005)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:40.922511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:40.978167 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f872791891f44303801e441459245005'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:40.990225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f872791891f44303801e441459245005'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:40.999181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f872791891f44303801e441459245005'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:40.999800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f872791891f44303801e441459245005'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:41.006993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f872791891f44303801e441459245005'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:41.013605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1307/2547] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 20:59:40 2022] DELETE /identity/v3/users/f872791891f44303801e441459245005 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:41.018193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbc6ff-2fbf-4539-a94b-868a401028d7 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:41.018686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbc6ff-2fbf-4539-a94b-868a401028d7 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:41.018792 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbc6ff-2fbf-4539-a94b-868a401028d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:41.037657 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-78fbc6ff-2fbf-4539-a94b-868a401028d7 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:41.038951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-78fbc6ff-2fbf-4539-a94b-868a401028d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:41.104796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1241/2548] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:41.115143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:41.116582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:41.153091 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:41.177017 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05979a-5bfc-41f5-a405-0270de90e2a8 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:41.180641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0001a-1ea0-4abb-a1c3-38cafe2b5390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:41.181339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0001a-1ea0-4abb-a1c3-38cafe2b5390 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:41.181685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0001a-1ea0-4abb-a1c3-38cafe2b5390 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:41.182006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0001a-1ea0-4abb-a1c3-38cafe2b5390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:41.192064 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0001a-1ea0-4abb-a1c3-38cafe2b5390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:41.193545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0001a-1ea0-4abb-a1c3-38cafe2b5390 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:41.202908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1308/2549] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:59:41.502265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbc6ff-2fbf-4539-a94b-868a401028d7 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:41.504705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-78fbc6ff-2fbf-4539-a94b-868a401028d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:41.521122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbc6ff-2fbf-4539-a94b-868a401028d7 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:41.528129 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:41.529278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:41.529758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:41.530043 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] PATH_INFO: `/v3/projects/fc46a5fc6c9240ad9234a97d66627fb9` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:41.531438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc46a5fc6c9240ad9234a97d66627fb9)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:41.533675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:41.535482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:41.591192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc46a5fc6c9240ad9234a97d66627fb9', 'request_id': 'req-5a9e8060-053d-4395-9551-34a3b9eccfa3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:41.598538 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc46a5fc6c9240ad9234a97d66627fb9', 'request_id': 'req-5a9e8060-053d-4395-9551-34a3b9eccfa3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:41.604518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc46a5fc6c9240ad9234a97d66627fb9', 'request_id': 'req-5a9e8060-053d-4395-9551-34a3b9eccfa3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:41.619137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] The token cache is being invalidate because project fc46a5fc6c9240ad9234a97d66627fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:59:41.619137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:41.619450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-5a9e8060-053d-4395-9551-34a3b9eccfa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:41.620880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1242/2550] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 20:59:41 2022] DELETE /identity/v3/projects/fc46a5fc6c9240ad9234a97d66627fb9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:41.626058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d8a0e-bb01-47c7-ad6b-afb4be3be3d8 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:41.626577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d8a0e-bb01-47c7-ad6b-afb4be3be3d8 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:41.626685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d8a0e-bb01-47c7-ad6b-afb4be3be3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:41.644678 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a53d8a0e-bb01-47c7-ad6b-afb4be3be3d8 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:41.645937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a53d8a0e-bb01-47c7-ad6b-afb4be3be3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:41.707372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1309/2551] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:41.715519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b3481-5cd3-4eb9-bf4d-236bdf5e1c40 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:41.715805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b3481-5cd3-4eb9-bf4d-236bdf5e1c40 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:41.716079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b3481-5cd3-4eb9-bf4d-236bdf5e1c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:41.736091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-5d1b3481-5cd3-4eb9-bf4d-236bdf5e1c40 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:41.737328 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5d1b3481-5cd3-4eb9-bf4d-236bdf5e1c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:41.745474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1243/2552] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:59:41.749840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d8a0e-bb01-47c7-ad6b-afb4be3be3d8 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:41.759532 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d8a0e-bb01-47c7-ad6b-afb4be3be3d8 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:41.764194 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91935a-300a-4e5c-82ec-0e6fac7215c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:41.764851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91935a-300a-4e5c-82ec-0e6fac7215c4 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:41.765133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91935a-300a-4e5c-82ec-0e6fac7215c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:41.765445 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91935a-300a-4e5c-82ec-0e6fac7215c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:41.766174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91935a-300a-4e5c-82ec-0e6fac7215c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:59:41.766511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91935a-300a-4e5c-82ec-0e6fac7215c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:41.768091 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91935a-300a-4e5c-82ec-0e6fac7215c4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:41.779931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1310/2553] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 20:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:59:41.784309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b3481-5cd3-4eb9-bf4d-236bdf5e1c40 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:41.794079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b3481-5cd3-4eb9-bf4d-236bdf5e1c40 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:41.798772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83fed8-979c-4897-85a4-f2a5ff334d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:41.799418 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83fed8-979c-4897-85a4-f2a5ff334d49 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:41.799701 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83fed8-979c-4897-85a4-f2a5ff334d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:41.799974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83fed8-979c-4897-85a4-f2a5ff334d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:41.800659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83fed8-979c-4897-85a4-f2a5ff334d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:41.802589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83fed8-979c-4897-85a4-f2a5ff334d49 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:41.828387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1244/2554] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:59:41 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:41.832718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91935a-300a-4e5c-82ec-0e6fac7215c4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:41.843336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91935a-300a-4e5c-82ec-0e6fac7215c4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:41.901771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299899-6e46-47bf-a36a-7f588a937dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:41.903132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299899-6e46-47bf-a36a-7f588a937dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:41.903417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299899-6e46-47bf-a36a-7f588a937dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:41.903722 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299899-6e46-47bf-a36a-7f588a937dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:41.904387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8299899-6e46-47bf-a36a-7f588a937dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:41.906323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8299899-6e46-47bf-a36a-7f588a937dc9 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:41.933698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1311/2555] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:59:41 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:41.935171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83fed8-979c-4897-85a4-f2a5ff334d49 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:41.946721 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83fed8-979c-4897-85a4-f2a5ff334d49 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:41.952035 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46912b-6251-4abb-aebb-da5bcbb12c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:41.952913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46912b-6251-4abb-aebb-da5bcbb12c70 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:41.953385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46912b-6251-4abb-aebb-da5bcbb12c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:41.953870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46912b-6251-4abb-aebb-da5bcbb12c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:41.954640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46912b-6251-4abb-aebb-da5bcbb12c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:41.957077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46912b-6251-4abb-aebb-da5bcbb12c70 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:41.977745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1245/2556] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:41.981142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299899-6e46-47bf-a36a-7f588a937dc9 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:41.991637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299899-6e46-47bf-a36a-7f588a937dc9 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:41.996682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb75e0-e69c-4e7a-8a31-45ee70ad53f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:41.997456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb75e0-e69c-4e7a-8a31-45ee70ad53f3 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:41.997741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb75e0-e69c-4e7a-8a31-45ee70ad53f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:41.998049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb75e0-e69c-4e7a-8a31-45ee70ad53f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:41.998779 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb75e0-e69c-4e7a-8a31-45ee70ad53f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:42.000354 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb75e0-e69c-4e7a-8a31-45ee70ad53f3 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:42.011181 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1312/2557] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:42.015814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46912b-6251-4abb-aebb-da5bcbb12c70 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:42.025866 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46912b-6251-4abb-aebb-da5bcbb12c70 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:42.032003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be01758-8011-4d56-b2c4-12be89112442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:42.033071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be01758-8011-4d56-b2c4-12be89112442 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:42.033566 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be01758-8011-4d56-b2c4-12be89112442 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:42.033839 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be01758-8011-4d56-b2c4-12be89112442 admin admin] PATH_INFO: `/v3/projects/4343340155a04c3390277edb1fd95862/users/fb00621b3e5343288039eebd603ac180/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:42.053034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be01758-8011-4d56-b2c4-12be89112442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4343340155a04c3390277edb1fd95862, user_id=fb00621b3e5343288039eebd603ac180, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:42.055957 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be01758-8011-4d56-b2c4-12be89112442 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:42.070686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1246/2558] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:42 2022] PUT /identity/v3/projects/4343340155a04c3390277edb1fd95862/users/fb00621b3e5343288039eebd603ac180/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:42.081769 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c32a3-4c17-47d2-a649-a06398f6eae5 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:42.082066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c32a3-4c17-47d2-a649-a06398f6eae5 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:42.082398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c32a3-4c17-47d2-a649-a06398f6eae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:42.110672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-fd9c32a3-4c17-47d2-a649-a06398f6eae5 None None] MFA Rules not processed for user `fb00621b3e5343288039eebd603ac180`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:42.111881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-fd9c32a3-4c17-47d2-a649-a06398f6eae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:42.142055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:42.178649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1313/2559] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:42.184178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be01758-8011-4d56-b2c4-12be89112442 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:42.245159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be01758-8011-4d56-b2c4-12be89112442 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:42.249893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b659eb-2f97-4613-97a1-fef218367b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:42.250554 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b659eb-2f97-4613-97a1-fef218367b8a admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:42.250840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b659eb-2f97-4613-97a1-fef218367b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:42.251118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b659eb-2f97-4613-97a1-fef218367b8a admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:42.251812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b659eb-2f97-4613-97a1-fef218367b8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:42.253740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b659eb-2f97-4613-97a1-fef218367b8a admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:42.281489 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1247/2560] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 20:59:42 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:42.285672 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c32a3-4c17-47d2-a649-a06398f6eae5 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:42.296667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c32a3-4c17-47d2-a649-a06398f6eae5 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:42.353964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f305af-736b-457a-afbb-5d957bc0d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:42.354641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f305af-736b-457a-afbb-5d957bc0d5ca admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:42.354930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f305af-736b-457a-afbb-5d957bc0d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:42.355209 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f305af-736b-457a-afbb-5d957bc0d5ca admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:42.355906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f305af-736b-457a-afbb-5d957bc0d5ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:42.357893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f305af-736b-457a-afbb-5d957bc0d5ca admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:42.379055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1314/2561] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 20:59:42 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:42.382885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b659eb-2f97-4613-97a1-fef218367b8a admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:42.393933 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b659eb-2f97-4613-97a1-fef218367b8a admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:42.398718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca1231-1550-4ecb-961e-0eaf1451b6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:42.399345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1231-1550-4ecb-961e-0eaf1451b6bc admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:42.399629 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1231-1550-4ecb-961e-0eaf1451b6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:42.399909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1231-1550-4ecb-961e-0eaf1451b6bc admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:42.400643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca1231-1550-4ecb-961e-0eaf1451b6bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:42.402294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca1231-1550-4ecb-961e-0eaf1451b6bc admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:42.407474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:42.414025 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1248/2562] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:42.417603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f305af-736b-457a-afbb-5d957bc0d5ca admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:42.427930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f305af-736b-457a-afbb-5d957bc0d5ca admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:42.432810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0730c4-f6b9-4fc2-803e-11970b2920fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:42.434224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0730c4-f6b9-4fc2-803e-11970b2920fb admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:42.434513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0730c4-f6b9-4fc2-803e-11970b2920fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:42.434793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0730c4-f6b9-4fc2-803e-11970b2920fb admin admin] PATH_INFO: `/v3/projects/336b63ab7f434545a20c38fb0b3d79ca/users/06e6ce01369f4a4dbefd635efc8c34a7/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:42.451489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0730c4-f6b9-4fc2-803e-11970b2920fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=336b63ab7f434545a20c38fb0b3d79ca, user_id=06e6ce01369f4a4dbefd635efc8c34a7, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:42.454215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0730c4-f6b9-4fc2-803e-11970b2920fb admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:42.468406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1315/2563] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:42 2022] PUT /identity/v3/projects/336b63ab7f434545a20c38fb0b3d79ca/users/06e6ce01369f4a4dbefd635efc8c34a7/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:42.472395 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca1231-1550-4ecb-961e-0eaf1451b6bc admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:42.483357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca1231-1550-4ecb-961e-0eaf1451b6bc admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:42.541763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3615a-a65f-4c3d-9a34-dc827331a047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:42.542438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3615a-a65f-4c3d-9a34-dc827331a047 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:42.542914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3615a-a65f-4c3d-9a34-dc827331a047 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:42.543192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3615a-a65f-4c3d-9a34-dc827331a047 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:42.543950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3615a-a65f-4c3d-9a34-dc827331a047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:42.545571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3615a-a65f-4c3d-9a34-dc827331a047 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:42.555431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1249/2564] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:42.560117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0730c4-f6b9-4fc2-803e-11970b2920fb admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:42.571202 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0730c4-f6b9-4fc2-803e-11970b2920fb admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:42.576641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c77a9e-ae18-4d62-aced-8b746231b0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:42.577572 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77a9e-ae18-4d62-aced-8b746231b0ed admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:42.577860 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77a9e-ae18-4d62-aced-8b746231b0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:42.578156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77a9e-ae18-4d62-aced-8b746231b0ed admin admin] PATH_INFO: `/v3/projects/336b63ab7f434545a20c38fb0b3d79ca/users/06e6ce01369f4a4dbefd635efc8c34a7/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:42.580144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c77a9e-ae18-4d62-aced-8b746231b0ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=336b63ab7f434545a20c38fb0b3d79ca, user_id=06e6ce01369f4a4dbefd635efc8c34a7, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:42.582836 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c77a9e-ae18-4d62-aced-8b746231b0ed admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:42.596822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1316/2565] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:42 2022] PUT /identity/v3/projects/336b63ab7f434545a20c38fb0b3d79ca/users/06e6ce01369f4a4dbefd635efc8c34a7/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:42.609177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518b878-aa10-48bd-bd45-865070298878 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:42.609494 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518b878-aa10-48bd-bd45-865070298878 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:42.609780 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518b878-aa10-48bd-bd45-865070298878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:42.638952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-e518b878-aa10-48bd-bd45-865070298878 None None] MFA Rules not processed for user `06e6ce01369f4a4dbefd635efc8c34a7`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:42.640184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e518b878-aa10-48bd-bd45-865070298878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:42.717846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1250/2566] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 20:59:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:42.731534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c77a9e-ae18-4d62-aced-8b746231b0ed admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:42.734638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-90c77a9e-ae18-4d62-aced-8b746231b0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:42.812902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c77a9e-ae18-4d62-aced-8b746231b0ed admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:42.817077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b36fe-0441-4a2a-b5af-dc734a11066c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:42.817785 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b36fe-0441-4a2a-b5af-dc734a11066c service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:42.818076 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b36fe-0441-4a2a-b5af-dc734a11066c service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:42.818355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b36fe-0441-4a2a-b5af-dc734a11066c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:42.831933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b36fe-0441-4a2a-b5af-dc734a11066c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:42.833308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b36fe-0441-4a2a-b5af-dc734a11066c service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:42.845174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1317/2567] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:59:42.943373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518b878-aa10-48bd-bd45-865070298878 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:42.953711 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518b878-aa10-48bd-bd45-865070298878 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:42.957330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55cc7-55ce-4b3c-a027-dc00d320379a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:42.958871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55cc7-55ce-4b3c-a027-dc00d320379a service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:42.958871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55cc7-55ce-4b3c-a027-dc00d320379a service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:42.958871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55cc7-55ce-4b3c-a027-dc00d320379a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:43.039675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a55cc7-55ce-4b3c-a027-dc00d320379a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:43.041205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a55cc7-55ce-4b3c-a027-dc00d320379a service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:43.051809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1251/2568] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:44.577401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b36fe-0441-4a2a-b5af-dc734a11066c service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:44.579253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-468b36fe-0441-4a2a-b5af-dc734a11066c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:44.592176 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b36fe-0441-4a2a-b5af-dc734a11066c service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:44.597680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085ae3c-6cbb-4c14-b889-22274c7ca557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:44.598942 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085ae3c-6cbb-4c14-b889-22274c7ca557 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:44.599240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085ae3c-6cbb-4c14-b889-22274c7ca557 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:44.599510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085ae3c-6cbb-4c14-b889-22274c7ca557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:44.601576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d085ae3c-6cbb-4c14-b889-22274c7ca557 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:44.640921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:44.708087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085ae3c-6cbb-4c14-b889-22274c7ca557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:44.708087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085ae3c-6cbb-4c14-b889-22274c7ca557 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:44.733159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1318/2569] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:53.047254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=91 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:54.718010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=18 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:08.616209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55cc7-55ce-4b3c-a027-dc00d320379a service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:08.618697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-82a55cc7-55ce-4b3c-a027-dc00d320379a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:08.724202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55cc7-55ce-4b3c-a027-dc00d320379a service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:08.730203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064bcd8-7c1e-491b-9d8f-068beec1674a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:08.731109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064bcd8-7c1e-491b-9d8f-068beec1674a service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:08.731602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064bcd8-7c1e-491b-9d8f-068beec1674a service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:08.732089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064bcd8-7c1e-491b-9d8f-068beec1674a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:08.734481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0064bcd8-7c1e-491b-9d8f-068beec1674a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:08.749595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064bcd8-7c1e-491b-9d8f-068beec1674a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:08.753053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064bcd8-7c1e-491b-9d8f-068beec1674a service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:08.772726 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1252/2570] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:14.635103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085ae3c-6cbb-4c14-b889-22274c7ca557 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:14.719198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085ae3c-6cbb-4c14-b889-22274c7ca557 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:14.720717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:14.721664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:14.721953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:14.722476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] PATH_INFO: `/v3/users/fb00621b3e5343288039eebd603ac180` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:14.726685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb00621b3e5343288039eebd603ac180)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:14.730692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:14.797348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb00621b3e5343288039eebd603ac180'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:14.812502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb00621b3e5343288039eebd603ac180'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:14.823191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb00621b3e5343288039eebd603ac180'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:14.824824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb00621b3e5343288039eebd603ac180'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:14.841151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb00621b3e5343288039eebd603ac180'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:14.858408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1319/2571] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:14 2022] DELETE /identity/v3/users/fb00621b3e5343288039eebd603ac180 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:14.861155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626f277-3306-4155-b6fe-57d82b1bd0cf None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:14.862287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626f277-3306-4155-b6fe-57d82b1bd0cf None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:14.862287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626f277-3306-4155-b6fe-57d82b1bd0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:14.888739 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-2626f277-3306-4155-b6fe-57d82b1bd0cf None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:14.890051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2626f277-3306-4155-b6fe-57d82b1bd0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:14.963665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1253/2572] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:14.974090 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:14.975652 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:15.039830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362cc76-8ff6-4318-964f-ddcf21fc5d18 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:15.044775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc1c2d-3a1f-4713-b2f5-90977de7e0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:15.045609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc1c2d-3a1f-4713-b2f5-90977de7e0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:15.045990 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc1c2d-3a1f-4713-b2f5-90977de7e0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:15.046371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc1c2d-3a1f-4713-b2f5-90977de7e0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:15.056821 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc1c2d-3a1f-4713-b2f5-90977de7e0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:15.058856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc1c2d-3a1f-4713-b2f5-90977de7e0c6 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:15.069879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1320/2573] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:00:15.382692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626f277-3306-4155-b6fe-57d82b1bd0cf None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:15.384575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2626f277-3306-4155-b6fe-57d82b1bd0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:15.405646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626f277-3306-4155-b6fe-57d82b1bd0cf None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:15.416491 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:15.418223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:15.418223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:15.418546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] PATH_INFO: `/v3/projects/4343340155a04c3390277edb1fd95862` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:15.420551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4343340155a04c3390277edb1fd95862)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:15.423640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:15.498118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4343340155a04c3390277edb1fd95862', 'request_id': 'req-7b853604-865e-4435-b7de-3c610b27d23e'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:15.507278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4343340155a04c3390277edb1fd95862', 'request_id': 'req-7b853604-865e-4435-b7de-3c610b27d23e'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:15.516293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4343340155a04c3390277edb1fd95862', 'request_id': 'req-7b853604-865e-4435-b7de-3c610b27d23e'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:15.533648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] The token cache is being invalidate because project 4343340155a04c3390277edb1fd95862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:15.534041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:15.534693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:15.536693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1254/2574] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:15 2022] DELETE /identity/v3/projects/4343340155a04c3390277edb1fd95862 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:15.541068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc1c2d-3a1f-4713-b2f5-90977de7e0c6 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:15.555863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc1c2d-3a1f-4713-b2f5-90977de7e0c6 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:15.583565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:15.649204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc102f5-019e-43f5-aee5-857260f77fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:15.650863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc102f5-019e-43f5-aee5-857260f77fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:15.651352 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc102f5-019e-43f5-aee5-857260f77fad admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:15.651826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc102f5-019e-43f5-aee5-857260f77fad admin admin] PATH_INFO: `/v3/users/06e6ce01369f4a4dbefd635efc8c34a7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:15.654068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc102f5-019e-43f5-aee5-857260f77fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e6ce01369f4a4dbefd635efc8c34a7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:15.657798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc102f5-019e-43f5-aee5-857260f77fad admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:15.719540 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-abc102f5-019e-43f5-aee5-857260f77fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e6ce01369f4a4dbefd635efc8c34a7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:15.734501 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-abc102f5-019e-43f5-aee5-857260f77fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e6ce01369f4a4dbefd635efc8c34a7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:15.743296 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-abc102f5-019e-43f5-aee5-857260f77fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e6ce01369f4a4dbefd635efc8c34a7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:15.743995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-abc102f5-019e-43f5-aee5-857260f77fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e6ce01369f4a4dbefd635efc8c34a7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:15.753329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-abc102f5-019e-43f5-aee5-857260f77fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e6ce01369f4a4dbefd635efc8c34a7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:15.760174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1321/2575] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:15 2022] DELETE /identity/v3/users/06e6ce01369f4a4dbefd635efc8c34a7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:16.119085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:16.121831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:16.141872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b853604-865e-4435-b7de-3c610b27d23e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:16.178760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:16.218670 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:16.219537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:16.219817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:16.220093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] PATH_INFO: `/v3/projects/336b63ab7f434545a20c38fb0b3d79ca` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:16.221294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=336b63ab7f434545a20c38fb0b3d79ca)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:16.223640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:16.279240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '336b63ab7f434545a20c38fb0b3d79ca', 'request_id': 'req-c1547af3-238e-4362-b3c9-fbe616ec2441'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:16.289442 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '336b63ab7f434545a20c38fb0b3d79ca', 'request_id': 'req-c1547af3-238e-4362-b3c9-fbe616ec2441'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:16.297165 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '336b63ab7f434545a20c38fb0b3d79ca', 'request_id': 'req-c1547af3-238e-4362-b3c9-fbe616ec2441'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:16.325522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] The token cache is being invalidate because project 336b63ab7f434545a20c38fb0b3d79ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:16.326006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:16.326998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-c1547af3-238e-4362-b3c9-fbe616ec2441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:16.329888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1255/2576] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:16 2022] DELETE /identity/v3/projects/336b63ab7f434545a20c38fb0b3d79ca => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:16.336907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d489f7-b8ba-4070-8875-ae3c318126b6 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:16.337236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d489f7-b8ba-4070-8875-ae3c318126b6 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:16.337592 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d489f7-b8ba-4070-8875-ae3c318126b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:16.363281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-62d489f7-b8ba-4070-8875-ae3c318126b6 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:16.364502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-62d489f7-b8ba-4070-8875-ae3c318126b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:16.434459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1322/2577] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:16.442884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426881c-6178-4f14-bc1c-57522dc991bd None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:16.443185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426881c-6178-4f14-bc1c-57522dc991bd None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:16.443457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426881c-6178-4f14-bc1c-57522dc991bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:16.463346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-0426881c-6178-4f14-bc1c-57522dc991bd None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:16.465240 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0426881c-6178-4f14-bc1c-57522dc991bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:16.473812 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1256/2578] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:00:16.479025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d489f7-b8ba-4070-8875-ae3c318126b6 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:16.494724 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d489f7-b8ba-4070-8875-ae3c318126b6 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:16.500049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75a665-8b77-4bc9-9b7b-051c7d172f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:16.500713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a665-8b77-4bc9-9b7b-051c7d172f63 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:16.501001 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a665-8b77-4bc9-9b7b-051c7d172f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:16.501281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a665-8b77-4bc9-9b7b-051c7d172f63 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:16.502055 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75a665-8b77-4bc9-9b7b-051c7d172f63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:00:16.502362 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75a665-8b77-4bc9-9b7b-051c7d172f63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:16.503974 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75a665-8b77-4bc9-9b7b-051c7d172f63 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:16.516518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1323/2579] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:00:16.520851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426881c-6178-4f14-bc1c-57522dc991bd None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:16.530386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426881c-6178-4f14-bc1c-57522dc991bd None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:16.535293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36e906-0555-4d55-b81b-fcf70746096f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:16.535952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e906-0555-4d55-b81b-fcf70746096f admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:16.536233 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e906-0555-4d55-b81b-fcf70746096f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:16.536507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e906-0555-4d55-b81b-fcf70746096f admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:16.537202 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36e906-0555-4d55-b81b-fcf70746096f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:16.539134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36e906-0555-4d55-b81b-fcf70746096f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:16.575518 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1257/2580] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:16 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:16.582099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75a665-8b77-4bc9-9b7b-051c7d172f63 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:16.600649 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75a665-8b77-4bc9-9b7b-051c7d172f63 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:16.630035 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:16.665851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd65b3d-03c4-40f4-86e7-e742fdd828a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:16.666949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd65b3d-03c4-40f4-86e7-e742fdd828a6 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:16.667805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd65b3d-03c4-40f4-86e7-e742fdd828a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:16.668183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd65b3d-03c4-40f4-86e7-e742fdd828a6 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:16.668975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd65b3d-03c4-40f4-86e7-e742fdd828a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:16.671048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd65b3d-03c4-40f4-86e7-e742fdd828a6 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:16.695906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1324/2581] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:16 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:16.698913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36e906-0555-4d55-b81b-fcf70746096f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:16.711327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36e906-0555-4d55-b81b-fcf70746096f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:16.717054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9038e31-f7b2-4f80-b113-392a4425fecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:16.717776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9038e31-f7b2-4f80-b113-392a4425fecc admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:16.718084 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9038e31-f7b2-4f80-b113-392a4425fecc admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:16.718356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9038e31-f7b2-4f80-b113-392a4425fecc admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:16.719096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9038e31-f7b2-4f80-b113-392a4425fecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:16.720738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9038e31-f7b2-4f80-b113-392a4425fecc admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:16.733124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1258/2582] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:00:16.737488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd65b3d-03c4-40f4-86e7-e742fdd828a6 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:16.747833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd65b3d-03c4-40f4-86e7-e742fdd828a6 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:16.752459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0343c-8337-4f8c-8951-175b5bd2ec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:16.753126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0343c-8337-4f8c-8951-175b5bd2ec31 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:16.753438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0343c-8337-4f8c-8951-175b5bd2ec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:16.753720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0343c-8337-4f8c-8951-175b5bd2ec31 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:16.754448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0343c-8337-4f8c-8951-175b5bd2ec31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:16.756023 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0343c-8337-4f8c-8951-175b5bd2ec31 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:16.766500 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1325/2583] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:16.770769 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9038e31-f7b2-4f80-b113-392a4425fecc admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:16.782039 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9038e31-f7b2-4f80-b113-392a4425fecc admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:16.787084 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32b0be-dc52-4698-8c22-c7854797776c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:16.787974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b0be-dc52-4698-8c22-c7854797776c admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:16.788253 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b0be-dc52-4698-8c22-c7854797776c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:16.788526 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b0be-dc52-4698-8c22-c7854797776c admin admin] PATH_INFO: `/v3/projects/3e4a2628d8c4491e91330fdf4aef5ac1/users/7c0fc717635e4da68c3604e57235741c/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:16.805933 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32b0be-dc52-4698-8c22-c7854797776c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4a2628d8c4491e91330fdf4aef5ac1, user_id=7c0fc717635e4da68c3604e57235741c, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:16.808581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32b0be-dc52-4698-8c22-c7854797776c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:16.822875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1259/2584] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:16 2022] PUT /identity/v3/projects/3e4a2628d8c4491e91330fdf4aef5ac1/users/7c0fc717635e4da68c3604e57235741c/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:16.836733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f22ec9-8a92-4982-a715-f5d0fef7d317 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:16.837028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f22ec9-8a92-4982-a715-f5d0fef7d317 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:16.837314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f22ec9-8a92-4982-a715-f5d0fef7d317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:16.869571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-d2f22ec9-8a92-4982-a715-f5d0fef7d317 None None] MFA Rules not processed for user `7c0fc717635e4da68c3604e57235741c`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:16.871031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d2f22ec9-8a92-4982-a715-f5d0fef7d317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:16.950059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1326/2585] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:16.954988 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32b0be-dc52-4698-8c22-c7854797776c admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:17.030860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:17.047297 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32b0be-dc52-4698-8c22-c7854797776c admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:17.052365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8d5e1-7a16-44a1-9022-b0037d101fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:17.053042 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8d5e1-7a16-44a1-9022-b0037d101fec admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:17.053354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8d5e1-7a16-44a1-9022-b0037d101fec admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:17.053661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8d5e1-7a16-44a1-9022-b0037d101fec admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:17.054447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8d5e1-7a16-44a1-9022-b0037d101fec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:17.056393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8d5e1-7a16-44a1-9022-b0037d101fec admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:17.085804 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1260/2586] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:16 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:17.090573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f22ec9-8a92-4982-a715-f5d0fef7d317 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:17.103738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f22ec9-8a92-4982-a715-f5d0fef7d317 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:17.167819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e5343-c437-4d46-a9fc-ff192dd5ca7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:17.168528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e5343-c437-4d46-a9fc-ff192dd5ca7f admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:17.168822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e5343-c437-4d46-a9fc-ff192dd5ca7f admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:17.169108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e5343-c437-4d46-a9fc-ff192dd5ca7f admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:17.169862 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e5343-c437-4d46-a9fc-ff192dd5ca7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:17.171809 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e5343-c437-4d46-a9fc-ff192dd5ca7f admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:17.192580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1327/2587] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:17 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:17.196690 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8d5e1-7a16-44a1-9022-b0037d101fec admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:17.208730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8d5e1-7a16-44a1-9022-b0037d101fec admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:17.213334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d5cd-27bc-49de-a288-991493461c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:17.214133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d5cd-27bc-49de-a288-991493461c37 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:17.214949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d5cd-27bc-49de-a288-991493461c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:17.215226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d5cd-27bc-49de-a288-991493461c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:17.215956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d5cd-27bc-49de-a288-991493461c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:17.217588 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d5cd-27bc-49de-a288-991493461c37 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:17.229718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1261/2588] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:17.234045 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e5343-c437-4d46-a9fc-ff192dd5ca7f admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:17.244613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e5343-c437-4d46-a9fc-ff192dd5ca7f admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:17.249723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2bd8e-ffed-4a77-b9f3-8782514ebd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:17.250655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bd8e-ffed-4a77-b9f3-8782514ebd75 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:17.250935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bd8e-ffed-4a77-b9f3-8782514ebd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:17.251210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2bd8e-ffed-4a77-b9f3-8782514ebd75 admin admin] PATH_INFO: `/v3/projects/002aea0a59de4b59965ce5550565c2ec/users/3e047539dfd44fda986a10da008f7934/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:17.269031 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2bd8e-ffed-4a77-b9f3-8782514ebd75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=002aea0a59de4b59965ce5550565c2ec, user_id=3e047539dfd44fda986a10da008f7934, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:17.271683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2bd8e-ffed-4a77-b9f3-8782514ebd75 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:17.286819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1328/2589] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:17 2022] PUT /identity/v3/projects/002aea0a59de4b59965ce5550565c2ec/users/3e047539dfd44fda986a10da008f7934/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:17.292402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d5cd-27bc-49de-a288-991493461c37 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:17.303004 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d5cd-27bc-49de-a288-991493461c37 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:17.368852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e20c91-b324-4ffa-8335-24434d506ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:17.369813 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e20c91-b324-4ffa-8335-24434d506ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:17.370131 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e20c91-b324-4ffa-8335-24434d506ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:17.370416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e20c91-b324-4ffa-8335-24434d506ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:17.371164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e20c91-b324-4ffa-8335-24434d506ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:17.372798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e20c91-b324-4ffa-8335-24434d506ef2 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:17.384063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1262/2590] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:17.390667 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2bd8e-ffed-4a77-b9f3-8782514ebd75 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:17.396265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:17.403084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2bd8e-ffed-4a77-b9f3-8782514ebd75 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:17.409346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899719b9-ad72-425a-a536-e802c87b00f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:17.410271 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899719b9-ad72-425a-a536-e802c87b00f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:17.410545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899719b9-ad72-425a-a536-e802c87b00f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:17.410818 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899719b9-ad72-425a-a536-e802c87b00f4 admin admin] PATH_INFO: `/v3/projects/002aea0a59de4b59965ce5550565c2ec/users/3e047539dfd44fda986a10da008f7934/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:17.412822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899719b9-ad72-425a-a536-e802c87b00f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=002aea0a59de4b59965ce5550565c2ec, user_id=3e047539dfd44fda986a10da008f7934, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:17.415508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899719b9-ad72-425a-a536-e802c87b00f4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:17.427517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1329/2591] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:17 2022] PUT /identity/v3/projects/002aea0a59de4b59965ce5550565c2ec/users/3e047539dfd44fda986a10da008f7934/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:17.441595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df7e86-a76f-4ee0-a423-a08a5fa084dd None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:17.441909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df7e86-a76f-4ee0-a423-a08a5fa084dd None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:17.442188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df7e86-a76f-4ee0-a423-a08a5fa084dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:17.474257 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-b0df7e86-a76f-4ee0-a423-a08a5fa084dd None None] MFA Rules not processed for user `3e047539dfd44fda986a10da008f7934`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:17.475484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b0df7e86-a76f-4ee0-a423-a08a5fa084dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:17.552036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1263/2592] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:17.563618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899719b9-ad72-425a-a536-e802c87b00f4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:17.565494 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-899719b9-ad72-425a-a536-e802c87b00f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:17.624638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899719b9-ad72-425a-a536-e802c87b00f4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:17.628594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11361c2c-2264-44b8-9522-6c23187e8308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:17.629248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11361c2c-2264-44b8-9522-6c23187e8308 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:17.629559 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11361c2c-2264-44b8-9522-6c23187e8308 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:17.629835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11361c2c-2264-44b8-9522-6c23187e8308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:17.702947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11361c2c-2264-44b8-9522-6c23187e8308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:17.704631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11361c2c-2264-44b8-9522-6c23187e8308 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:17.716126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1330/2593] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:00:27.548678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:27.711332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:29.338967 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df7e86-a76f-4ee0-a423-a08a5fa084dd None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:29.341080 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b0df7e86-a76f-4ee0-a423-a08a5fa084dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:29.353253 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df7e86-a76f-4ee0-a423-a08a5fa084dd None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:29.357122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66d2e0-491c-4b10-ae93-ece3a139dfa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:29.357844 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66d2e0-491c-4b10-ae93-ece3a139dfa3 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:29.358159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66d2e0-491c-4b10-ae93-ece3a139dfa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:29.358430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66d2e0-491c-4b10-ae93-ece3a139dfa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:29.368732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66d2e0-491c-4b10-ae93-ece3a139dfa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:29.370208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66d2e0-491c-4b10-ae93-ece3a139dfa3 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:29.384653 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1264/2594] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:30.910417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11361c2c-2264-44b8-9522-6c23187e8308 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:30.912143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-11361c2c-2264-44b8-9522-6c23187e8308 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:31.024943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11361c2c-2264-44b8-9522-6c23187e8308 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:31.034882 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:31.035777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:31.036063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:31.036347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] PATH_INFO: `/v3/users/e7f8113c231d4c2499630b2f6c83a07a` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:31.037900 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f8113c231d4c2499630b2f6c83a07a)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:31.040203 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:31.103190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f8113c231d4c2499630b2f6c83a07a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.121499 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f8113c231d4c2499630b2f6c83a07a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.133334 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f8113c231d4c2499630b2f6c83a07a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.134589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f8113c231d4c2499630b2f6c83a07a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.149062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f8113c231d4c2499630b2f6c83a07a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.160040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1331/2595] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:30 2022] DELETE /identity/v3/users/e7f8113c231d4c2499630b2f6c83a07a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:31.167461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3826f-1423-445c-89e5-8cd5971508be None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:31.168390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3826f-1423-445c-89e5-8cd5971508be None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:31.168390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3826f-1423-445c-89e5-8cd5971508be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:31.192519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-c9b3826f-1423-445c-89e5-8cd5971508be None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:31.193997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c9b3826f-1423-445c-89e5-8cd5971508be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:31.304498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1265/2596] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:31.315039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:31.316477 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:31.390887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b160bb-95f3-43ec-87a0-3d07dbc4d4ca admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:31.400146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed0061-59c7-4a10-a7f5-f19574db918e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:31.400851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed0061-59c7-4a10-a7f5-f19574db918e service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:31.401104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed0061-59c7-4a10-a7f5-f19574db918e service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:31.401417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed0061-59c7-4a10-a7f5-f19574db918e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:31.412709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed0061-59c7-4a10-a7f5-f19574db918e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:31.414190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed0061-59c7-4a10-a7f5-f19574db918e service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:31.421630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3826f-1423-445c-89e5-8cd5971508be None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:31.423223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c9b3826f-1423-445c-89e5-8cd5971508be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:31.425071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1332/2597] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:31.436040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3826f-1423-445c-89e5-8cd5971508be None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:31.443484 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:31.444975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:31.445532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:31.446013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] PATH_INFO: `/v3/users/39d780775b2f4f9098d8655dfa233a50` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:31.448242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d780775b2f4f9098d8655dfa233a50)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:31.452532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:31.525760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d780775b2f4f9098d8655dfa233a50'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.536752 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d780775b2f4f9098d8655dfa233a50'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.547949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d780775b2f4f9098d8655dfa233a50'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.564497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d780775b2f4f9098d8655dfa233a50'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.565481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d780775b2f4f9098d8655dfa233a50'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.573622 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1266/2598] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:31 2022] DELETE /identity/v3/users/39d780775b2f4f9098d8655dfa233a50 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:31.578544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a66a0-31a4-49d0-9a53-a84a530c22bd None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:31.578798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a66a0-31a4-49d0-9a53-a84a530c22bd None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:31.579073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a66a0-31a4-49d0-9a53-a84a530c22bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:31.600190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0c1a66a0-31a4-49d0-9a53-a84a530c22bd None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:31.601530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0c1a66a0-31a4-49d0-9a53-a84a530c22bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:31.627237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:31.681419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1333/2599] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:31.691957 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:31.694461 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:31.721137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a66a0-31a4-49d0-9a53-a84a530c22bd None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:31.723419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0c1a66a0-31a4-49d0-9a53-a84a530c22bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:31.736806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a66a0-31a4-49d0-9a53-a84a530c22bd None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:31.741586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:31.742713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:31.743068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:31.743450 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] PATH_INFO: `/v3/projects/1100541a02904558b72f70d1483d9425` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:31.745218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1100541a02904558b72f70d1483d9425)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:31.750774 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:31.762437 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb914-e292-4625-9278-f4e907b973e6 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:31.767151 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62794c0e-3c22-4928-90f1-fc0ebb5f941d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:31.767876 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62794c0e-3c22-4928-90f1-fc0ebb5f941d service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:31.768251 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62794c0e-3c22-4928-90f1-fc0ebb5f941d service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:31.768530 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62794c0e-3c22-4928-90f1-fc0ebb5f941d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:31.778022 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62794c0e-3c22-4928-90f1-fc0ebb5f941d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:31.779478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62794c0e-3c22-4928-90f1-fc0ebb5f941d service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:31.790434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1267/2600] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:31.803419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1100541a02904558b72f70d1483d9425', 'request_id': 'req-77b40b91-5624-4a04-befd-a81e42bc93bd'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.813969 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1100541a02904558b72f70d1483d9425', 'request_id': 'req-77b40b91-5624-4a04-befd-a81e42bc93bd'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.819949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1100541a02904558b72f70d1483d9425', 'request_id': 'req-77b40b91-5624-4a04-befd-a81e42bc93bd'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.832633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] The token cache is being invalidate because project 1100541a02904558b72f70d1483d9425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:31.832932 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.833625 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.835188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1334/2601] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:31 2022] DELETE /identity/v3/projects/1100541a02904558b72f70d1483d9425 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:31.839155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62794c0e-3c22-4928-90f1-fc0ebb5f941d service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:31.844554 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:31.851897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62794c0e-3c22-4928-90f1-fc0ebb5f941d service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:31.912636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:31.913626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:31.913851 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:31.914124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] PATH_INFO: `/v3/users/6a144ad0d31748719d03311406e02cb0` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:31.915377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a144ad0d31748719d03311406e02cb0)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:31.917330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:31.968948 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a144ad0d31748719d03311406e02cb0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.977114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a144ad0d31748719d03311406e02cb0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.984684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a144ad0d31748719d03311406e02cb0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.998834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a144ad0d31748719d03311406e02cb0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:31.999585 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a144ad0d31748719d03311406e02cb0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.007187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1268/2602] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:31 2022] DELETE /identity/v3/users/6a144ad0d31748719d03311406e02cb0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:32.082429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:32.084476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:32.103103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b40b91-5624-4a04-befd-a81e42bc93bd admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:32.206177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:32.207108 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.207502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.207692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] PATH_INFO: `/v3/projects/14ca5d3c21cd4a309b4f48f947bebf6b` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.208925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] RBAC: Authorizing `identity:delete_project(project_id=14ca5d3c21cd4a309b4f48f947bebf6b)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:32.210835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:32.260872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:32.262737 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14ca5d3c21cd4a309b4f48f947bebf6b', 'request_id': 'req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.272686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14ca5d3c21cd4a309b4f48f947bebf6b', 'request_id': 'req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.280375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14ca5d3c21cd4a309b4f48f947bebf6b', 'request_id': 'req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.291957 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] The token cache is being invalidate because project 14ca5d3c21cd4a309b4f48f947bebf6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:32.292253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.292930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.295259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1335/2603] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:32 2022] DELETE /identity/v3/projects/14ca5d3c21cd4a309b4f48f947bebf6b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:32.299041 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:32.300570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:32.315092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aca87f-d049-41ca-9db5-12f949368854 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:32.336339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:32.338656 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:32.399306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e1d78-e38e-4527-b330-bea5206879fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:32.400233 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e1d78-e38e-4527-b330-bea5206879fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.400588 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e1d78-e38e-4527-b330-bea5206879fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.400897 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e1d78-e38e-4527-b330-bea5206879fc admin admin] PATH_INFO: `/v3/users/67574db120b74be1a120bc65dbf603f8` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.402219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e1d78-e38e-4527-b330-bea5206879fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=67574db120b74be1a120bc65dbf603f8)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:32.404825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e1d78-e38e-4527-b330-bea5206879fc admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:32.428582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9aa1c-6259-4a29-b7aa-452ca0ee818a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:32.429496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:32.434434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:32.436114 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.436399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.436681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] PATH_INFO: `/v3/projects/9791dde84a804e73ac854a0339da3600` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.439735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9791dde84a804e73ac854a0339da3600)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:32.442155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:32.459633 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b58e1d78-e38e-4527-b330-bea5206879fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67574db120b74be1a120bc65dbf603f8'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.468184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b58e1d78-e38e-4527-b330-bea5206879fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67574db120b74be1a120bc65dbf603f8'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.476004 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b58e1d78-e38e-4527-b330-bea5206879fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67574db120b74be1a120bc65dbf603f8'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.490459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b58e1d78-e38e-4527-b330-bea5206879fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67574db120b74be1a120bc65dbf603f8'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.491176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-b58e1d78-e38e-4527-b330-bea5206879fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67574db120b74be1a120bc65dbf603f8'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.498653 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1269/2604] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:32 2022] DELETE /identity/v3/users/67574db120b74be1a120bc65dbf603f8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:32.498838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9791dde84a804e73ac854a0339da3600', 'request_id': 'req-c3633d87-146c-4721-ada3-5f564127d8f8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.509848 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9791dde84a804e73ac854a0339da3600', 'request_id': 'req-c3633d87-146c-4721-ada3-5f564127d8f8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.516175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9791dde84a804e73ac854a0339da3600', 'request_id': 'req-c3633d87-146c-4721-ada3-5f564127d8f8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.527348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] The token cache is being invalidate because project 9791dde84a804e73ac854a0339da3600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:32.527633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.528294 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-c3633d87-146c-4721-ada3-5f564127d8f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.530316 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1336/2605] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:32 2022] DELETE /identity/v3/projects/9791dde84a804e73ac854a0339da3600 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:32.535031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a533c91-605d-4354-9e13-df884e2ce2f3 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.537699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a533c91-605d-4354-9e13-df884e2ce2f3 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.538209 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a533c91-605d-4354-9e13-df884e2ce2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.558385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-2a533c91-605d-4354-9e13-df884e2ce2f3 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:32.560322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-2a533c91-605d-4354-9e13-df884e2ce2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:32.642316 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1270/2606] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:32.651096 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542e2ac-16f1-4e37-889c-75de029eb38d None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.651542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542e2ac-16f1-4e37-889c-75de029eb38d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.651960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542e2ac-16f1-4e37-889c-75de029eb38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.678211 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-7542e2ac-16f1-4e37-889c-75de029eb38d None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:32.679545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-7542e2ac-16f1-4e37-889c-75de029eb38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:32.689314 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1337/2607] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:32.693858 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a533c91-605d-4354-9e13-df884e2ce2f3 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:32.705401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a533c91-605d-4354-9e13-df884e2ce2f3 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:32.710554 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bfab4-c7bd-4bef-b28c-bff385a8534d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:32.711226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bfab4-c7bd-4bef-b28c-bff385a8534d admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.711513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bfab4-c7bd-4bef-b28c-bff385a8534d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.711789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bfab4-c7bd-4bef-b28c-bff385a8534d admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.713054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bfab4-c7bd-4bef-b28c-bff385a8534d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:00:32.713210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bfab4-c7bd-4bef-b28c-bff385a8534d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:32.715002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bfab4-c7bd-4bef-b28c-bff385a8534d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:32.728692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1271/2608] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:00:32.733633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542e2ac-16f1-4e37-889c-75de029eb38d None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:32.744655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542e2ac-16f1-4e37-889c-75de029eb38d None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:32.751221 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3d418-36c8-4887-9f91-475b58b26b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:32.751991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3d418-36c8-4887-9f91-475b58b26b0a admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.752348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3d418-36c8-4887-9f91-475b58b26b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.752711 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3d418-36c8-4887-9f91-475b58b26b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.753555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3d418-36c8-4887-9f91-475b58b26b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:32.755469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3d418-36c8-4887-9f91-475b58b26b0a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:32.790365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bfab4-c7bd-4bef-b28c-bff385a8534d admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:32.793286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1338/2609] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:32 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:32.794720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3d418-36c8-4887-9f91-475b58b26b0a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:32.795833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1d5bfab4-c7bd-4bef-b28c-bff385a8534d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:32.810729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bfab4-c7bd-4bef-b28c-bff385a8534d admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:32.845029 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:32.867117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3d418-36c8-4887-9f91-475b58b26b0a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:32.871571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:32.871743 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ede81-547d-4a60-90cf-45a1573c017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:32.872940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ede81-547d-4a60-90cf-45a1573c017b admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.873082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.873326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ede81-547d-4a60-90cf-45a1573c017b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.873495 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.873619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ede81-547d-4a60-90cf-45a1573c017b admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.873717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] PATH_INFO: `/v3/projects/5d115fcd8a4640449fa1160138057570` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.874232 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ede81-547d-4a60-90cf-45a1573c017b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:32.874809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d115fcd8a4640449fa1160138057570)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:32.876143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ede81-547d-4a60-90cf-45a1573c017b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:32.876695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:32.900955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1339/2610] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:32 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:00:32.902021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ede81-547d-4a60-90cf-45a1573c017b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:32.913015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ede81-547d-4a60-90cf-45a1573c017b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:32.917295 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:32.919830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e93595-490f-471d-b60d-cca429de7936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:32.920583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e93595-490f-471d-b60d-cca429de7936 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.920883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e93595-490f-471d-b60d-cca429de7936 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.921139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e93595-490f-471d-b60d-cca429de7936 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.921907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e93595-490f-471d-b60d-cca429de7936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:32.923576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e93595-490f-471d-b60d-cca429de7936 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:32.933612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d115fcd8a4640449fa1160138057570', 'request_id': 'req-8f10c06e-bfff-430a-9301-4479bfb02432'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.933878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1340/2611] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:00:32.938485 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e93595-490f-471d-b60d-cca429de7936 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:32.942454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d115fcd8a4640449fa1160138057570', 'request_id': 'req-8f10c06e-bfff-430a-9301-4479bfb02432'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.948447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e93595-490f-471d-b60d-cca429de7936 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:32.949124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d115fcd8a4640449fa1160138057570', 'request_id': 'req-8f10c06e-bfff-430a-9301-4479bfb02432'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.964053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] The token cache is being invalidate because project 5d115fcd8a4640449fa1160138057570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:32.964356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.965012 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-8f10c06e-bfff-430a-9301-4479bfb02432 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:32.967195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1272/2612] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:32 2022] DELETE /identity/v3/projects/5d115fcd8a4640449fa1160138057570 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:32.972121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f5450-6ec6-445c-81a6-ba4cdb8c7816 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.972418 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f5450-6ec6-445c-81a6-ba4cdb8c7816 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.972704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f5450-6ec6-445c-81a6-ba4cdb8c7816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.991325 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-743f5450-6ec6-445c-81a6-ba4cdb8c7816 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:32.992564 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-743f5450-6ec6-445c-81a6-ba4cdb8c7816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:33.011189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07605c89-e29f-4309-92f1-981e87fc0832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.012029 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07605c89-e29f-4309-92f1-981e87fc0832 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.012315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07605c89-e29f-4309-92f1-981e87fc0832 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.012593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07605c89-e29f-4309-92f1-981e87fc0832 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.013923 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07605c89-e29f-4309-92f1-981e87fc0832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.015525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07605c89-e29f-4309-92f1-981e87fc0832 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.027213 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1341/2613] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:00:33.032363 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07605c89-e29f-4309-92f1-981e87fc0832 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.034245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-07605c89-e29f-4309-92f1-981e87fc0832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:33.047217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07605c89-e29f-4309-92f1-981e87fc0832 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.052451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77119ed8-45c1-46fe-88a1-b45311f6574c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.053286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77119ed8-45c1-46fe-88a1-b45311f6574c admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.053604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77119ed8-45c1-46fe-88a1-b45311f6574c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.053891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77119ed8-45c1-46fe-88a1-b45311f6574c admin admin] PATH_INFO: `/v3/projects/61947a87c91e4c06b9c2671ba43c6c28/users/daf15efb9f6042ef90d58ec6bc838a61/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.054118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1273/2614] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:00:33.062483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c39a9-bf5e-4c57-a7d2-6a2470a59c86 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.062806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c39a9-bf5e-4c57-a7d2-6a2470a59c86 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.063609 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c39a9-bf5e-4c57-a7d2-6a2470a59c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.077681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77119ed8-45c1-46fe-88a1-b45311f6574c admin admin] RBAC: Authorizing `identity:create_grant(project_id=61947a87c91e4c06b9c2671ba43c6c28, user_id=daf15efb9f6042ef90d58ec6bc838a61, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.080455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77119ed8-45c1-46fe-88a1-b45311f6574c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.095770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-282c39a9-bf5e-4c57-a7d2-6a2470a59c86 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:33.097872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-282c39a9-bf5e-4c57-a7d2-6a2470a59c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:33.109514 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1342/2615] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:33 2022] PUT /identity/v3/projects/61947a87c91e4c06b9c2671ba43c6c28/users/daf15efb9f6042ef90d58ec6bc838a61/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:33.113646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cb2a0-8864-4a67-804c-de74b8fbfb12 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.113941 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cb2a0-8864-4a67-804c-de74b8fbfb12 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.114226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cb2a0-8864-4a67-804c-de74b8fbfb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.144785 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-8c5cb2a0-8864-4a67-804c-de74b8fbfb12 None None] MFA Rules not processed for user `daf15efb9f6042ef90d58ec6bc838a61`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:33.146605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8c5cb2a0-8864-4a67-804c-de74b8fbfb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:33.159545 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1274/2616] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:00:33.163906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c39a9-bf5e-4c57-a7d2-6a2470a59c86 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.173079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c39a9-bf5e-4c57-a7d2-6a2470a59c86 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.177926 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c39f9e-4167-4ab3-b6c6-403cb9d826ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.178639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c39f9e-4167-4ab3-b6c6-403cb9d826ae admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.179029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c39f9e-4167-4ab3-b6c6-403cb9d826ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.179205 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c39f9e-4167-4ab3-b6c6-403cb9d826ae admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.179903 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c39f9e-4167-4ab3-b6c6-403cb9d826ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:00:33.180204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c39f9e-4167-4ab3-b6c6-403cb9d826ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.181845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c39f9e-4167-4ab3-b6c6-403cb9d826ae admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.197695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1275/2617] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 21:00:33.198845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c39f9e-4167-4ab3-b6c6-403cb9d826ae admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.209255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c39f9e-4167-4ab3-b6c6-403cb9d826ae admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.214135 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9420c4-e34a-4f40-af42-a0286976b7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.214791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9420c4-e34a-4f40-af42-a0286976b7cf admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.215350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9420c4-e34a-4f40-af42-a0286976b7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.215640 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9420c4-e34a-4f40-af42-a0286976b7cf admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.216223 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1343/2618] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:33.216370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9420c4-e34a-4f40-af42-a0286976b7cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.218305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9420c4-e34a-4f40-af42-a0286976b7cf admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.220800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cb2a0-8864-4a67-804c-de74b8fbfb12 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.232957 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cb2a0-8864-4a67-804c-de74b8fbfb12 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.239021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42495edb-a3c7-4685-bff9-f19513e22ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.239786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42495edb-a3c7-4685-bff9-f19513e22ccd admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.240125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42495edb-a3c7-4685-bff9-f19513e22ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.240408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42495edb-a3c7-4685-bff9-f19513e22ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.241106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42495edb-a3c7-4685-bff9-f19513e22ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.243781 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42495edb-a3c7-4685-bff9-f19513e22ccd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.253228 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1276/2619] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:33 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:00:33.254637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9420c4-e34a-4f40-af42-a0286976b7cf admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.262071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:33.279806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1344/2620] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:33 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:33.281428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42495edb-a3c7-4685-bff9-f19513e22ccd admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.286793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:33.338477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9420c4-e34a-4f40-af42-a0286976b7cf admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.356768 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42495edb-a3c7-4685-bff9-f19513e22ccd admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.361758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0f0a5-a829-4351-85c2-963463de120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.362435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0f0a5-a829-4351-85c2-963463de120e admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.362726 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0f0a5-a829-4351-85c2-963463de120e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.363008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0f0a5-a829-4351-85c2-963463de120e admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.363704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0f0a5-a829-4351-85c2-963463de120e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.365761 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0f0a5-a829-4351-85c2-963463de120e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.390791 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1345/2621] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:33 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:00:33.394337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0f0a5-a829-4351-85c2-963463de120e admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.404512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0f0a5-a829-4351-85c2-963463de120e admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.409739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a948f7b-8af8-4227-a43b-361bf634fe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.410479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a948f7b-8af8-4227-a43b-361bf634fe0b admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.410840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a948f7b-8af8-4227-a43b-361bf634fe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.411321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a948f7b-8af8-4227-a43b-361bf634fe0b admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.412045 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a948f7b-8af8-4227-a43b-361bf634fe0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.413759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a948f7b-8af8-4227-a43b-361bf634fe0b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.418989 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fc346-3de0-4351-98e7-6f713474c902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.420385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fc346-3de0-4351-98e7-6f713474c902 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.420673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fc346-3de0-4351-98e7-6f713474c902 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.420994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fc346-3de0-4351-98e7-6f713474c902 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.421785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fc346-3de0-4351-98e7-6f713474c902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.423675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fc346-3de0-4351-98e7-6f713474c902 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.424565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1346/2622] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:00:33.428718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a948f7b-8af8-4227-a43b-361bf634fe0b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.440554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a948f7b-8af8-4227-a43b-361bf634fe0b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.448494 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1277/2623] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:33 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:00:33.448664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e01f8-9d22-4f6e-9ddf-32b34bb458e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.448664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e01f8-9d22-4f6e-9ddf-32b34bb458e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.448664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e01f8-9d22-4f6e-9ddf-32b34bb458e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.448664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e01f8-9d22-4f6e-9ddf-32b34bb458e4 admin admin] PATH_INFO: `/v3/projects/665845c5b216414293767bbda98abbda/users/d661a563958c49eea7d83d0f7b55d81c/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.449710 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fc346-3de0-4351-98e7-6f713474c902 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.461885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fc346-3de0-4351-98e7-6f713474c902 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.465476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e01f8-9d22-4f6e-9ddf-32b34bb458e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=665845c5b216414293767bbda98abbda, user_id=d661a563958c49eea7d83d0f7b55d81c, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.467392 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1848b19-5275-491e-9bcd-b5dcdb248c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.468283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1848b19-5275-491e-9bcd-b5dcdb248c2f admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.468843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1848b19-5275-491e-9bcd-b5dcdb248c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.468987 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e01f8-9d22-4f6e-9ddf-32b34bb458e4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.469303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1848b19-5275-491e-9bcd-b5dcdb248c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.470709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1848b19-5275-491e-9bcd-b5dcdb248c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.472940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1848b19-5275-491e-9bcd-b5dcdb248c2f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.484178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1347/2624] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:33 2022] PUT /identity/v3/projects/665845c5b216414293767bbda98abbda/users/d661a563958c49eea7d83d0f7b55d81c/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:33.485949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1278/2625] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:33.489056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e01f8-9d22-4f6e-9ddf-32b34bb458e4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.490424 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1848b19-5275-491e-9bcd-b5dcdb248c2f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.501850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1848b19-5275-491e-9bcd-b5dcdb248c2f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.558151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e01f8-9d22-4f6e-9ddf-32b34bb458e4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.564222 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16227c-18d6-4f4f-9a5c-2fd77fef9d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.565015 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16227c-18d6-4f4f-9a5c-2fd77fef9d36 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.565309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16227c-18d6-4f4f-9a5c-2fd77fef9d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.565621 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16227c-18d6-4f4f-9a5c-2fd77fef9d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.566412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16227c-18d6-4f4f-9a5c-2fd77fef9d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.568135 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16227c-18d6-4f4f-9a5c-2fd77fef9d36 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.582172 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1348/2626] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:33.582381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab65c88-3c43-4fbf-a94f-2c56de839820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.582381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65c88-3c43-4fbf-a94f-2c56de839820 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.582381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65c88-3c43-4fbf-a94f-2c56de839820 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.582381 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65c88-3c43-4fbf-a94f-2c56de839820 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.582800 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab65c88-3c43-4fbf-a94f-2c56de839820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.582970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab65c88-3c43-4fbf-a94f-2c56de839820 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.583525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16227c-18d6-4f4f-9a5c-2fd77fef9d36 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.593916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16227c-18d6-4f4f-9a5c-2fd77fef9d36 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.600059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1279/2627] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:00:33.600341 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a447870-e11c-4876-a34d-259224327846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.601056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab65c88-3c43-4fbf-a94f-2c56de839820 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.601248 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a447870-e11c-4876-a34d-259224327846 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.601489 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a447870-e11c-4876-a34d-259224327846 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.601764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a447870-e11c-4876-a34d-259224327846 admin admin] PATH_INFO: `/v3/projects/665845c5b216414293767bbda98abbda/users/d661a563958c49eea7d83d0f7b55d81c/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.605540 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a447870-e11c-4876-a34d-259224327846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=665845c5b216414293767bbda98abbda, user_id=d661a563958c49eea7d83d0f7b55d81c, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.611170 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a447870-e11c-4876-a34d-259224327846 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.612116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab65c88-3c43-4fbf-a94f-2c56de839820 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.616850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27be53c-611a-4ad7-9f6b-d2f8678012c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.617785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27be53c-611a-4ad7-9f6b-d2f8678012c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.618070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27be53c-611a-4ad7-9f6b-d2f8678012c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.618357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27be53c-611a-4ad7-9f6b-d2f8678012c3 admin admin] PATH_INFO: `/v3/projects/eebc8a5fe2a34d9d8965ebcc3543ed3d/users/74b9894abc3d46eaa00237be93b9a056/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.631066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1349/2628] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:33 2022] PUT /identity/v3/projects/665845c5b216414293767bbda98abbda/users/d661a563958c49eea7d83d0f7b55d81c/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:33.634436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27be53c-611a-4ad7-9f6b-d2f8678012c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eebc8a5fe2a34d9d8965ebcc3543ed3d, user_id=74b9894abc3d46eaa00237be93b9a056, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.637108 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27be53c-611a-4ad7-9f6b-d2f8678012c3 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.650620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d8439-95a1-40d1-806d-3d624f3d24c0 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.651224 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d8439-95a1-40d1-806d-3d624f3d24c0 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.651517 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d8439-95a1-40d1-806d-3d624f3d24c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.652361 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1280/2629] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:33 2022] PUT /identity/v3/projects/eebc8a5fe2a34d9d8965ebcc3543ed3d/users/74b9894abc3d46eaa00237be93b9a056/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:33.671724 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7a54a-5b7d-4794-9c84-4e046e7a4f85 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.672017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7a54a-5b7d-4794-9c84-4e046e7a4f85 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.672299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7a54a-5b7d-4794-9c84-4e046e7a4f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.689418 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-ce4d8439-95a1-40d1-806d-3d624f3d24c0 None None] MFA Rules not processed for user `d661a563958c49eea7d83d0f7b55d81c`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:33.690685 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ce4d8439-95a1-40d1-806d-3d624f3d24c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:33.702422 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-75b7a54a-5b7d-4794-9c84-4e046e7a4f85 None None] MFA Rules not processed for user `74b9894abc3d46eaa00237be93b9a056`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:33.703742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-75b7a54a-5b7d-4794-9c84-4e046e7a4f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:33.713009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:33.724016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:33.768688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1350/2630] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:33 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:33.779332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1281/2631] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:00:33.780149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d8439-95a1-40d1-806d-3d624f3d24c0 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.781912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ce4d8439-95a1-40d1-806d-3d624f3d24c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:33.790738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7a54a-5b7d-4794-9c84-4e046e7a4f85 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.792174 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-75b7a54a-5b7d-4794-9c84-4e046e7a4f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:33.844330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d8439-95a1-40d1-806d-3d624f3d24c0 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.848623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f686b-a5f2-410c-9396-5610c94ff5cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.849295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f686b-a5f2-410c-9396-5610c94ff5cd service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.849603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f686b-a5f2-410c-9396-5610c94ff5cd service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.849884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f686b-a5f2-410c-9396-5610c94ff5cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.860932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7a54a-5b7d-4794-9c84-4e046e7a4f85 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.863427 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f686b-a5f2-410c-9396-5610c94ff5cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.864783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b92b1-76e9-4c80-8b05-3be33d94e2b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.864914 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f686b-a5f2-410c-9396-5610c94ff5cd service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.865474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b92b1-76e9-4c80-8b05-3be33d94e2b4 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.865808 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b92b1-76e9-4c80-8b05-3be33d94e2b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.866080 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b92b1-76e9-4c80-8b05-3be33d94e2b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:33.878823 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b92b1-76e9-4c80-8b05-3be33d94e2b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:33.880272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b92b1-76e9-4c80-8b05-3be33d94e2b4 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:33.888454 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1351/2632] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:33 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:00:33.891727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1282/2633] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:00:33.921123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f686b-a5f2-410c-9396-5610c94ff5cd service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.939055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b92b1-76e9-4c80-8b05-3be33d94e2b4 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:33.949610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b92b1-76e9-4c80-8b05-3be33d94e2b4 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:33.956601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cbbf5-31ac-4426-a2ba-df3591d2addd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:33.956601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cbbf5-31ac-4426-a2ba-df3591d2addd service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:33.956601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cbbf5-31ac-4426-a2ba-df3591d2addd service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:33.956601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cbbf5-31ac-4426-a2ba-df3591d2addd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.016419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f686b-a5f2-410c-9396-5610c94ff5cd service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:34.022561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:34.023759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:34.024229 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:34.024620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] PATH_INFO: `/v3/users/74b9894abc3d46eaa00237be93b9a056` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.026138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b9894abc3d46eaa00237be93b9a056)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:34.029047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:34.033742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cbbf5-31ac-4426-a2ba-df3591d2addd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:34.035194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cbbf5-31ac-4426-a2ba-df3591d2addd service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:34.045774 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1283/2634] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:00:34.086253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b9894abc3d46eaa00237be93b9a056'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.099981 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:34.101659 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b9894abc3d46eaa00237be93b9a056'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.115287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b9894abc3d46eaa00237be93b9a056'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.116355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b9894abc3d46eaa00237be93b9a056'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.123922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b9894abc3d46eaa00237be93b9a056'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.130318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1352/2635] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:33 2022] DELETE /identity/v3/users/74b9894abc3d46eaa00237be93b9a056 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:34.135333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e80881-2dbe-487c-a628-ac6750c41b67 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:34.135727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e80881-2dbe-487c-a628-ac6750c41b67 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:34.136062 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e80881-2dbe-487c-a628-ac6750c41b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.165335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-21e80881-2dbe-487c-a628-ac6750c41b67 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:34.166575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-21e80881-2dbe-487c-a628-ac6750c41b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:34.222746 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:34.237816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1284/2636] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:34.248560 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:34.250087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:34.314402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86d5ff-1dde-4561-b44e-5fd12014c115 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:34.319003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a2cec-8677-4838-9d63-722bdbaba057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:34.319849 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a2cec-8677-4838-9d63-722bdbaba057 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:34.320195 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a2cec-8677-4838-9d63-722bdbaba057 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:34.320533 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a2cec-8677-4838-9d63-722bdbaba057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.331725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a2cec-8677-4838-9d63-722bdbaba057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:34.333160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a2cec-8677-4838-9d63-722bdbaba057 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:34.344777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1353/2637] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:34.391417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e80881-2dbe-487c-a628-ac6750c41b67 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:34.392786 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-21e80881-2dbe-487c-a628-ac6750c41b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:34.408529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e80881-2dbe-487c-a628-ac6750c41b67 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:34.416908 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:34.418128 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:34.418467 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:34.418791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] PATH_INFO: `/v3/users/7c0fc717635e4da68c3604e57235741c` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.420421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0fc717635e4da68c3604e57235741c)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:34.422725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:34.488576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0fc717635e4da68c3604e57235741c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.497531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0fc717635e4da68c3604e57235741c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.506353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0fc717635e4da68c3604e57235741c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.521342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0fc717635e4da68c3604e57235741c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.521342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0fc717635e4da68c3604e57235741c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.526715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1285/2638] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:34 2022] DELETE /identity/v3/users/7c0fc717635e4da68c3604e57235741c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:34.532173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57845-667f-49b8-b432-7d2705c447a5 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:34.532469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57845-667f-49b8-b432-7d2705c447a5 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:34.532750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57845-667f-49b8-b432-7d2705c447a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.550702 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-35d57845-667f-49b8-b432-7d2705c447a5 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:34.551930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-35d57845-667f-49b8-b432-7d2705c447a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:34.618380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1354/2639] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:34.628029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:34.629557 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:34.661237 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57845-667f-49b8-b432-7d2705c447a5 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:34.662708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-35d57845-667f-49b8-b432-7d2705c447a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:34.675222 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57845-667f-49b8-b432-7d2705c447a5 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:34.680173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:34.681040 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:34.681704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:34.681704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] PATH_INFO: `/v3/projects/eebc8a5fe2a34d9d8965ebcc3543ed3d` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.682873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eebc8a5fe2a34d9d8965ebcc3543ed3d)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:34.685308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:34.703391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e2704-70ec-4bc2-80dc-f12f5d3e4b5e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:34.707743 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41688a-76cf-45dc-9e8e-881e69096c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:34.708481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41688a-76cf-45dc-9e8e-881e69096c22 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:34.708845 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41688a-76cf-45dc-9e8e-881e69096c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:34.709194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41688a-76cf-45dc-9e8e-881e69096c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.719354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41688a-76cf-45dc-9e8e-881e69096c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:34.721006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41688a-76cf-45dc-9e8e-881e69096c22 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:34.731522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1286/2640] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:00:34.735741 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eebc8a5fe2a34d9d8965ebcc3543ed3d', 'request_id': 'req-6191434d-9fb0-4858-90ba-5f4e983bf3c6'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.744536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eebc8a5fe2a34d9d8965ebcc3543ed3d', 'request_id': 'req-6191434d-9fb0-4858-90ba-5f4e983bf3c6'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.751502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eebc8a5fe2a34d9d8965ebcc3543ed3d', 'request_id': 'req-6191434d-9fb0-4858-90ba-5f4e983bf3c6'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.763951 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] The token cache is being invalidate because project eebc8a5fe2a34d9d8965ebcc3543ed3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:34.764246 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.765006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6191434d-9fb0-4858-90ba-5f4e983bf3c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:34.766698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1355/2641] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:34 2022] DELETE /identity/v3/projects/eebc8a5fe2a34d9d8965ebcc3543ed3d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:34.771806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0be62-5857-495d-bc61-42b3cf4f5d55 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:34.772100 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0be62-5857-495d-bc61-42b3cf4f5d55 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:34.772448 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0be62-5857-495d-bc61-42b3cf4f5d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.791168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ddc0be62-5857-495d-bc61-42b3cf4f5d55 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:34.792391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ddc0be62-5857-495d-bc61-42b3cf4f5d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:34.858550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1287/2642] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:34.866279 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8465ffb-ee59-4818-9d91-69f0862eef00 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:34.866574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8465ffb-ee59-4818-9d91-69f0862eef00 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:34.866857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8465ffb-ee59-4818-9d91-69f0862eef00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.881196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:34.889964 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-b8465ffb-ee59-4818-9d91-69f0862eef00 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:34.891632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b8465ffb-ee59-4818-9d91-69f0862eef00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:34.901054 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1356/2643] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:34.905317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0be62-5857-495d-bc61-42b3cf4f5d55 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:34.916356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0be62-5857-495d-bc61-42b3cf4f5d55 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:34.922348 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3dc04-7b29-4a05-9f43-9a476ad22842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:34.923028 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3dc04-7b29-4a05-9f43-9a476ad22842 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:34.923314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3dc04-7b29-4a05-9f43-9a476ad22842 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:34.923594 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3dc04-7b29-4a05-9f43-9a476ad22842 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.924334 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3dc04-7b29-4a05-9f43-9a476ad22842 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:00:34.924645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3dc04-7b29-4a05-9f43-9a476ad22842 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:34.926298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3dc04-7b29-4a05-9f43-9a476ad22842 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:34.929105 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:34.941507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1288/2644] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:00:34.946350 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8465ffb-ee59-4818-9d91-69f0862eef00 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:34.957917 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8465ffb-ee59-4818-9d91-69f0862eef00 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:34.962885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553be6d-7517-451c-b0b2-e423719fb9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:34.963546 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553be6d-7517-451c-b0b2-e423719fb9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:34.963826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553be6d-7517-451c-b0b2-e423719fb9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:34.964100 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553be6d-7517-451c-b0b2-e423719fb9a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:34.964805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553be6d-7517-451c-b0b2-e423719fb9a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:34.966814 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553be6d-7517-451c-b0b2-e423719fb9a2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:34.996479 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1357/2645] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:34 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:00:35.002430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3dc04-7b29-4a05-9f43-9a476ad22842 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:35.021879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3dc04-7b29-4a05-9f43-9a476ad22842 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:35.041968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553be6d-7517-451c-b0b2-e423719fb9a2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:35.043753 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3553be6d-7517-451c-b0b2-e423719fb9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:35.108152 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b352ce2-d549-4b32-9bdf-b7185d9e4347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:35.109016 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b352ce2-d549-4b32-9bdf-b7185d9e4347 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:35.109320 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b352ce2-d549-4b32-9bdf-b7185d9e4347 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:35.109634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b352ce2-d549-4b32-9bdf-b7185d9e4347 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:35.110336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b352ce2-d549-4b32-9bdf-b7185d9e4347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:35.113589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b352ce2-d549-4b32-9bdf-b7185d9e4347 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:35.123928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553be6d-7517-451c-b0b2-e423719fb9a2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:35.129278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:35.130326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:35.130615 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:35.130894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] PATH_INFO: `/v3/projects/3e4a2628d8c4491e91330fdf4aef5ac1` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:35.132138 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e4a2628d8c4491e91330fdf4aef5ac1)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:35.134111 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:35.139172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1289/2646] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:34 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:35.140691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b352ce2-d549-4b32-9bdf-b7185d9e4347 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:35.152059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b352ce2-d549-4b32-9bdf-b7185d9e4347 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:35.160196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf07bc-7013-4ab8-86b3-ab393d66428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:35.161042 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf07bc-7013-4ab8-86b3-ab393d66428f admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:35.161454 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf07bc-7013-4ab8-86b3-ab393d66428f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:35.161830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf07bc-7013-4ab8-86b3-ab393d66428f admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:35.162825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf07bc-7013-4ab8-86b3-ab393d66428f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:35.164716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf07bc-7013-4ab8-86b3-ab393d66428f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:35.181136 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1290/2647] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:35.182653 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf07bc-7013-4ab8-86b3-ab393d66428f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:35.190247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e4a2628d8c4491e91330fdf4aef5ac1', 'request_id': 'req-a934fc0c-f7f6-49b4-a650-48737ecee01b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:35.194615 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf07bc-7013-4ab8-86b3-ab393d66428f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:35.204746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e4a2628d8c4491e91330fdf4aef5ac1', 'request_id': 'req-a934fc0c-f7f6-49b4-a650-48737ecee01b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:35.211150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e4a2628d8c4491e91330fdf4aef5ac1', 'request_id': 'req-a934fc0c-f7f6-49b4-a650-48737ecee01b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:35.223880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] The token cache is being invalidate because project 3e4a2628d8c4491e91330fdf4aef5ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:35.224332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:35.225025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:35.226633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1358/2648] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:35 2022] DELETE /identity/v3/projects/3e4a2628d8c4491e91330fdf4aef5ac1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:35.230455 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:35.232293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:35.261798 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ff6a9-b0d7-4bf0-a4a4-eef1d9ce3e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:35.262380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ff6a9-b0d7-4bf0-a4a4-eef1d9ce3e86 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:35.262663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ff6a9-b0d7-4bf0-a4a4-eef1d9ce3e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:35.263115 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ff6a9-b0d7-4bf0-a4a4-eef1d9ce3e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:35.263846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ff6a9-b0d7-4bf0-a4a4-eef1d9ce3e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:35.265475 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ff6a9-b0d7-4bf0-a4a4-eef1d9ce3e86 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:35.278977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1291/2649] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:35.279943 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ff6a9-b0d7-4bf0-a4a4-eef1d9ce3e86 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:35.281816 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-6a4ff6a9-b0d7-4bf0-a4a4-eef1d9ce3e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:35.293957 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ff6a9-b0d7-4bf0-a4a4-eef1d9ce3e86 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:35.299189 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460d92c-8b74-490c-84fd-b834d19b0c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:35.300070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d92c-8b74-490c-84fd-b834d19b0c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:35.300396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d92c-8b74-490c-84fd-b834d19b0c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:35.300566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934fc0c-f7f6-49b4-a650-48737ecee01b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:35.300686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d92c-8b74-490c-84fd-b834d19b0c94 admin admin] PATH_INFO: `/v3/projects/94975be780694882b1689446ae402965/users/792d034bc1fd424e9c00356c2b6eb2f6/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:35.304965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:35.305857 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:35.306178 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:35.306437 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] PATH_INFO: `/v3/users/3e047539dfd44fda986a10da008f7934` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:35.307629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e047539dfd44fda986a10da008f7934)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:35.309556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:35.316692 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460d92c-8b74-490c-84fd-b834d19b0c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94975be780694882b1689446ae402965, user_id=792d034bc1fd424e9c00356c2b6eb2f6, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:35.319372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460d92c-8b74-490c-84fd-b834d19b0c94 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:35.321874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:35.334982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1292/2650] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:35 2022] PUT /identity/v3/projects/94975be780694882b1689446ae402965/users/792d034bc1fd424e9c00356c2b6eb2f6/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:35.346570 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20778969-a4e2-4193-bd2d-9312419c2e15 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:35.347098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20778969-a4e2-4193-bd2d-9312419c2e15 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:35.347372 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20778969-a4e2-4193-bd2d-9312419c2e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:35.370765 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e047539dfd44fda986a10da008f7934'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:35.385618 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-20778969-a4e2-4193-bd2d-9312419c2e15 None None] MFA Rules not processed for user `792d034bc1fd424e9c00356c2b6eb2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:35.386280 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-20778969-a4e2-4193-bd2d-9312419c2e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:35.391725 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e047539dfd44fda986a10da008f7934'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:35.401010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e047539dfd44fda986a10da008f7934'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:35.402168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e047539dfd44fda986a10da008f7934'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:35.411257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e047539dfd44fda986a10da008f7934'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:35.417587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1359/2651] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:35 2022] DELETE /identity/v3/users/3e047539dfd44fda986a10da008f7934 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:35.428842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:35.469769 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1293/2652] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:00:35.482285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:35.486066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:35.559908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a94a6-0e71-4ff2-9a80-2441270f4f55 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:35.563947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d90b4-6c1a-4ed9-993e-9b7725b1484c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:35.564752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d90b4-6c1a-4ed9-993e-9b7725b1484c service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:35.565118 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d90b4-6c1a-4ed9-993e-9b7725b1484c service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:35.565512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d90b4-6c1a-4ed9-993e-9b7725b1484c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:35.646439 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d90b4-6c1a-4ed9-993e-9b7725b1484c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:35.648004 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d90b4-6c1a-4ed9-993e-9b7725b1484c service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:35.659475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1360/2653] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:35.736397 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20778969-a4e2-4193-bd2d-9312419c2e15 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:35.737951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-20778969-a4e2-4193-bd2d-9312419c2e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:35.853950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20778969-a4e2-4193-bd2d-9312419c2e15 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:35.952032 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:35.952950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:35.953231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:35.953549 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] PATH_INFO: `/v3/projects/002aea0a59de4b59965ce5550565c2ec` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:35.954827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] RBAC: Authorizing `identity:delete_project(project_id=002aea0a59de4b59965ce5550565c2ec)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:35.956804 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:36.026909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '002aea0a59de4b59965ce5550565c2ec', 'request_id': 'req-799cdc48-0c07-4889-a90c-4bf4daab9eee'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:36.034289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '002aea0a59de4b59965ce5550565c2ec', 'request_id': 'req-799cdc48-0c07-4889-a90c-4bf4daab9eee'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:36.041688 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '002aea0a59de4b59965ce5550565c2ec', 'request_id': 'req-799cdc48-0c07-4889-a90c-4bf4daab9eee'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:36.055682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] The token cache is being invalidate because project 002aea0a59de4b59965ce5550565c2ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:36.055972 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:36.056601 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-799cdc48-0c07-4889-a90c-4bf4daab9eee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:36.058423 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1294/2654] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:35 2022] DELETE /identity/v3/projects/002aea0a59de4b59965ce5550565c2ec => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:36.065025 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a50b0-7720-4529-9402-890fa3543013 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:36.065336 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a50b0-7720-4529-9402-890fa3543013 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:36.065746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a50b0-7720-4529-9402-890fa3543013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:36.085182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-700a50b0-7720-4529-9402-890fa3543013 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:36.086598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-700a50b0-7720-4529-9402-890fa3543013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:36.128366 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:36.151185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1361/2655] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:36.160045 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e593e1a-b3ea-4cb9-b64d-e9bc8d580aa6 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:36.160336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e593e1a-b3ea-4cb9-b64d-e9bc8d580aa6 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:36.160617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e593e1a-b3ea-4cb9-b64d-e9bc8d580aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:36.180520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-7e593e1a-b3ea-4cb9-b64d-e9bc8d580aa6 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:36.181770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7e593e1a-b3ea-4cb9-b64d-e9bc8d580aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:36.190074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1295/2656] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:36.194534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a50b0-7720-4529-9402-890fa3543013 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:36.206875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a50b0-7720-4529-9402-890fa3543013 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:36.211567 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85892a02-4b90-4dd0-8be5-181d3e5b1f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:36.212214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85892a02-4b90-4dd0-8be5-181d3e5b1f29 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:36.212497 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85892a02-4b90-4dd0-8be5-181d3e5b1f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:36.212771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85892a02-4b90-4dd0-8be5-181d3e5b1f29 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:36.213522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85892a02-4b90-4dd0-8be5-181d3e5b1f29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:00:36.213825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85892a02-4b90-4dd0-8be5-181d3e5b1f29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:36.215408 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85892a02-4b90-4dd0-8be5-181d3e5b1f29 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:36.227722 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1362/2657] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:00:36.232368 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e593e1a-b3ea-4cb9-b64d-e9bc8d580aa6 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:36.242676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e593e1a-b3ea-4cb9-b64d-e9bc8d580aa6 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:36.247473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42819c9-f694-4253-9d57-80e952a154d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:36.248132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42819c9-f694-4253-9d57-80e952a154d9 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:36.248415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42819c9-f694-4253-9d57-80e952a154d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:36.248691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42819c9-f694-4253-9d57-80e952a154d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:36.249410 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42819c9-f694-4253-9d57-80e952a154d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:36.251311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42819c9-f694-4253-9d57-80e952a154d9 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:36.254551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:36.279203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1296/2658] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:36 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:36.283307 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85892a02-4b90-4dd0-8be5-181d3e5b1f29 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:36.293193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85892a02-4b90-4dd0-8be5-181d3e5b1f29 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:36.349786 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb9720-dd58-4c8b-8cfc-b27666c92546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:36.350475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb9720-dd58-4c8b-8cfc-b27666c92546 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:36.350738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb9720-dd58-4c8b-8cfc-b27666c92546 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:36.351009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb9720-dd58-4c8b-8cfc-b27666c92546 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:36.351699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb9720-dd58-4c8b-8cfc-b27666c92546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:36.353610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb9720-dd58-4c8b-8cfc-b27666c92546 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:36.375391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1363/2659] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:36 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:36.379581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42819c9-f694-4253-9d57-80e952a154d9 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:36.389998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42819c9-f694-4253-9d57-80e952a154d9 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:36.394735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d04712-01a6-484a-b9a1-2ca92faaf9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:36.395399 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d04712-01a6-484a-b9a1-2ca92faaf9ef admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:36.395685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d04712-01a6-484a-b9a1-2ca92faaf9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:36.395968 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d04712-01a6-484a-b9a1-2ca92faaf9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:36.396715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d04712-01a6-484a-b9a1-2ca92faaf9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:36.398382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d04712-01a6-484a-b9a1-2ca92faaf9ef admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:36.409430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1297/2660] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:36.413525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb9720-dd58-4c8b-8cfc-b27666c92546 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:36.424083 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb9720-dd58-4c8b-8cfc-b27666c92546 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:36.428675 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc401d-e45b-472a-802e-170f3083c740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:36.429488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc401d-e45b-472a-802e-170f3083c740 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:36.429633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc401d-e45b-472a-802e-170f3083c740 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:36.429893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc401d-e45b-472a-802e-170f3083c740 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:36.430611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc401d-e45b-472a-802e-170f3083c740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:36.432438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc401d-e45b-472a-802e-170f3083c740 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:36.442613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1364/2661] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:00:36.446741 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d04712-01a6-484a-b9a1-2ca92faaf9ef admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:36.456919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d04712-01a6-484a-b9a1-2ca92faaf9ef admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:36.461647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ca836-58fa-4b1d-ac1d-fa428272e848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:36.462525 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ca836-58fa-4b1d-ac1d-fa428272e848 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:36.462814 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ca836-58fa-4b1d-ac1d-fa428272e848 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:36.463097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ca836-58fa-4b1d-ac1d-fa428272e848 admin admin] PATH_INFO: `/v3/projects/e24f214e7d5d4d32af0df7d71d616b26/users/80ce4b7a22f142c9a33cbd3b04e77cf0/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:36.478203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ca836-58fa-4b1d-ac1d-fa428272e848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24f214e7d5d4d32af0df7d71d616b26, user_id=80ce4b7a22f142c9a33cbd3b04e77cf0, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:36.480877 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ca836-58fa-4b1d-ac1d-fa428272e848 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:36.495208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1298/2662] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:36 2022] PUT /identity/v3/projects/e24f214e7d5d4d32af0df7d71d616b26/users/80ce4b7a22f142c9a33cbd3b04e77cf0/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:36.508402 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2577-9ec5-4ac9-ae44-32e4f36cd274 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:36.508927 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2577-9ec5-4ac9-ae44-32e4f36cd274 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:36.509212 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2577-9ec5-4ac9-ae44-32e4f36cd274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:36.536648 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-bf8f2577-9ec5-4ac9-ae44-32e4f36cd274 None None] MFA Rules not processed for user `80ce4b7a22f142c9a33cbd3b04e77cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:36.537833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-bf8f2577-9ec5-4ac9-ae44-32e4f36cd274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:36.605833 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1365/2663] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:36.618259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ca836-58fa-4b1d-ac1d-fa428272e848 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:36.619952 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-db7ca836-58fa-4b1d-ac1d-fa428272e848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:36.707149 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ca836-58fa-4b1d-ac1d-fa428272e848 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:36.712101 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541a1fe-d881-4cc7-8cc9-837ea2a0565d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82e1192b2d554199a08debb4aa392479', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:36.713286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541a1fe-d881-4cc7-8cc9-837ea2a0565d service glance] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:36.713626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541a1fe-d881-4cc7-8cc9-837ea2a0565d service glance] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:36.714026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541a1fe-d881-4cc7-8cc9-837ea2a0565d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:36.727785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541a1fe-d881-4cc7-8cc9-837ea2a0565d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:36.729817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541a1fe-d881-4cc7-8cc9-837ea2a0565d service glance] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:36.741665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1299/2664] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:37.604005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f2577-9ec5-4ac9-ae44-32e4f36cd274 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:37.605856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-bf8f2577-9ec5-4ac9-ae44-32e4f36cd274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:37.659593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:37.683827 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f2577-9ec5-4ac9-ae44-32e4f36cd274 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:37.694945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:37.695770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:37.696085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:37.696337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] PATH_INFO: `/v3/users/daf15efb9f6042ef90d58ec6bc838a61` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:37.697596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf15efb9f6042ef90d58ec6bc838a61)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:37.699542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:37.750696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf15efb9f6042ef90d58ec6bc838a61'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:37.763654 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf15efb9f6042ef90d58ec6bc838a61'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:37.771948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf15efb9f6042ef90d58ec6bc838a61'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:37.772574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf15efb9f6042ef90d58ec6bc838a61'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:37.779855 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf15efb9f6042ef90d58ec6bc838a61'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:37.785886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1366/2665] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:37 2022] DELETE /identity/v3/users/daf15efb9f6042ef90d58ec6bc838a61 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:37.790431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cb7df-18b7-4199-b2fc-5991b33eb9ac None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:37.790946 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cb7df-18b7-4199-b2fc-5991b33eb9ac None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:37.791069 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cb7df-18b7-4199-b2fc-5991b33eb9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:37.810705 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-854cb7df-18b7-4199-b2fc-5991b33eb9ac None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:37.811935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-854cb7df-18b7-4199-b2fc-5991b33eb9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:37.873147 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1300/2666] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:00:37.883281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:37.885000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:37.943740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35723d64-fd87-4cef-9a45-e640e017250a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:37.947420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5f1de-b652-4c83-8e29-bc6282385136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:37.948070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f1de-b652-4c83-8e29-bc6282385136 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:37.948359 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f1de-b652-4c83-8e29-bc6282385136 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:37.948636 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5f1de-b652-4c83-8e29-bc6282385136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:37.962027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5f1de-b652-4c83-8e29-bc6282385136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:37.963599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5f1de-b652-4c83-8e29-bc6282385136 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:37.974087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1367/2667] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:38.267738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854cb7df-18b7-4199-b2fc-5991b33eb9ac None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:38.269486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-854cb7df-18b7-4199-b2fc-5991b33eb9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:38.283960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854cb7df-18b7-4199-b2fc-5991b33eb9ac None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:38.291293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:38.292163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:38.292997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:38.293290 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] PATH_INFO: `/v3/projects/61947a87c91e4c06b9c2671ba43c6c28` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:38.295044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] RBAC: Authorizing `identity:delete_project(project_id=61947a87c91e4c06b9c2671ba43c6c28)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:38.297117 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:38.336807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:38.355357 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61947a87c91e4c06b9c2671ba43c6c28', 'request_id': 'req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.362237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61947a87c91e4c06b9c2671ba43c6c28', 'request_id': 'req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.369303 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61947a87c91e4c06b9c2671ba43c6c28', 'request_id': 'req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.386413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] The token cache is being invalidate because project 61947a87c91e4c06b9c2671ba43c6c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:38.386712 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.387365 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.389010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1301/2668] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:38 2022] DELETE /identity/v3/projects/61947a87c91e4c06b9c2671ba43c6c28 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:38.393175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5f1de-b652-4c83-8e29-bc6282385136 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:38.403199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5f1de-b652-4c83-8e29-bc6282385136 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:38.463955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ff090-c6cb-4256-8d4b-a34e584240c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:38.464838 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ff090-c6cb-4256-8d4b-a34e584240c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:38.465125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ff090-c6cb-4256-8d4b-a34e584240c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:38.465432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ff090-c6cb-4256-8d4b-a34e584240c2 admin admin] PATH_INFO: `/v3/users/d661a563958c49eea7d83d0f7b55d81c` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:38.467204 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ff090-c6cb-4256-8d4b-a34e584240c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d661a563958c49eea7d83d0f7b55d81c)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:38.469247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ff090-c6cb-4256-8d4b-a34e584240c2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:38.470783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:38.524835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a44ff090-c6cb-4256-8d4b-a34e584240c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd661a563958c49eea7d83d0f7b55d81c'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.537901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a44ff090-c6cb-4256-8d4b-a34e584240c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd661a563958c49eea7d83d0f7b55d81c'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.546601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a44ff090-c6cb-4256-8d4b-a34e584240c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd661a563958c49eea7d83d0f7b55d81c'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.547308 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a44ff090-c6cb-4256-8d4b-a34e584240c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd661a563958c49eea7d83d0f7b55d81c'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.554903 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a44ff090-c6cb-4256-8d4b-a34e584240c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd661a563958c49eea7d83d0f7b55d81c'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.561000 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1368/2669] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:38 2022] DELETE /identity/v3/users/d661a563958c49eea7d83d0f7b55d81c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:38.834352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:38.835930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:38.850173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0009dc-6a11-4736-aced-5a6bc2f8b95f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:38.911029 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:38.911821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:38.912113 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:38.912394 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] PATH_INFO: `/v3/projects/665845c5b216414293767bbda98abbda` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:38.913645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] RBAC: Authorizing `identity:delete_project(project_id=665845c5b216414293767bbda98abbda)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:38.915556 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:38.963157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '665845c5b216414293767bbda98abbda', 'request_id': 'req-ff7ad7fc-e309-4fde-befd-5c653e12a91b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.970085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '665845c5b216414293767bbda98abbda', 'request_id': 'req-ff7ad7fc-e309-4fde-befd-5c653e12a91b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.975859 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '665845c5b216414293767bbda98abbda', 'request_id': 'req-ff7ad7fc-e309-4fde-befd-5c653e12a91b'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.988671 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] The token cache is being invalidate because project 665845c5b216414293767bbda98abbda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:38.988954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.989608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ff7ad7fc-e309-4fde-befd-5c653e12a91b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:38.991144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1302/2670] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:38 2022] DELETE /identity/v3/projects/665845c5b216414293767bbda98abbda => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:38.996065 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5905df0-c65e-4579-b8e9-804f8e7f489e None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:38.996561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5905df0-c65e-4579-b8e9-804f8e7f489e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:38.996688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5905df0-c65e-4579-b8e9-804f8e7f489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.018396 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-e5905df0-c65e-4579-b8e9-804f8e7f489e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:39.020064 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e5905df0-c65e-4579-b8e9-804f8e7f489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:39.083159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1369/2671] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:39.091408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb680f9-2e39-4942-aa88-0e7ea74fcced None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.091702 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb680f9-2e39-4942-aa88-0e7ea74fcced None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.091979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb680f9-2e39-4942-aa88-0e7ea74fcced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.111402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-7fb680f9-2e39-4942-aa88-0e7ea74fcced None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:39.112653 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7fb680f9-2e39-4942-aa88-0e7ea74fcced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:39.120799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1303/2672] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:39.125075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5905df0-c65e-4579-b8e9-804f8e7f489e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:39.136348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5905df0-c65e-4579-b8e9-804f8e7f489e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:39.141902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d41ed5-7b0e-4fac-be61-e1c35e5fb89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:39.142575 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d41ed5-7b0e-4fac-be61-e1c35e5fb89e admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.142861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d41ed5-7b0e-4fac-be61-e1c35e5fb89e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.143149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d41ed5-7b0e-4fac-be61-e1c35e5fb89e admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.143887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d41ed5-7b0e-4fac-be61-e1c35e5fb89e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:00:39.144200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d41ed5-7b0e-4fac-be61-e1c35e5fb89e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:39.145912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d41ed5-7b0e-4fac-be61-e1c35e5fb89e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:39.158930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1370/2673] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:00:39.163342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb680f9-2e39-4942-aa88-0e7ea74fcced None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:39.173853 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb680f9-2e39-4942-aa88-0e7ea74fcced None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:39.178694 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8e65-765f-4628-8d70-2f5cfd9e1640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:39.179396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8e65-765f-4628-8d70-2f5cfd9e1640 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.179709 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8e65-765f-4628-8d70-2f5cfd9e1640 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.179986 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8e65-765f-4628-8d70-2f5cfd9e1640 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.180700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8e65-765f-4628-8d70-2f5cfd9e1640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:39.182740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb8e65-765f-4628-8d70-2f5cfd9e1640 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:39.213860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1304/2674] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:39 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:39.215071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d41ed5-7b0e-4fac-be61-e1c35e5fb89e admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:39.225797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d41ed5-7b0e-4fac-be61-e1c35e5fb89e admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:39.282437 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927444f-92aa-43b2-8b76-19a6a2934ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:39.283111 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927444f-92aa-43b2-8b76-19a6a2934ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.283398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927444f-92aa-43b2-8b76-19a6a2934ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.283676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927444f-92aa-43b2-8b76-19a6a2934ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.284361 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927444f-92aa-43b2-8b76-19a6a2934ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:39.286316 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927444f-92aa-43b2-8b76-19a6a2934ee9 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:39.295854 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:39.312459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1371/2675] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:39 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:39.313687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8e65-765f-4628-8d70-2f5cfd9e1640 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:39.324642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8e65-765f-4628-8d70-2f5cfd9e1640 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:39.329954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60babd7e-b375-435a-813d-4c14ef00b889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:39.330656 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60babd7e-b375-435a-813d-4c14ef00b889 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.330935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60babd7e-b375-435a-813d-4c14ef00b889 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.331210 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60babd7e-b375-435a-813d-4c14ef00b889 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.331950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60babd7e-b375-435a-813d-4c14ef00b889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:39.333560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60babd7e-b375-435a-813d-4c14ef00b889 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:39.344318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1305/2676] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:39.349193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927444f-92aa-43b2-8b76-19a6a2934ee9 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:39.360789 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927444f-92aa-43b2-8b76-19a6a2934ee9 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:39.365651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935ceea-20bd-4bd7-a4c2-2e401be317ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:39.366353 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ceea-20bd-4bd7-a4c2-2e401be317ea admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.366800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ceea-20bd-4bd7-a4c2-2e401be317ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.367084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ceea-20bd-4bd7-a4c2-2e401be317ea admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.367825 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935ceea-20bd-4bd7-a4c2-2e401be317ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:39.369465 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935ceea-20bd-4bd7-a4c2-2e401be317ea admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:39.382251 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1372/2677] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:39.386756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60babd7e-b375-435a-813d-4c14ef00b889 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:39.400132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60babd7e-b375-435a-813d-4c14ef00b889 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:39.405006 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e799863-2895-4441-8015-2e39f0ef6797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:39.406304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e799863-2895-4441-8015-2e39f0ef6797 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.406304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e799863-2895-4441-8015-2e39f0ef6797 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.406539 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e799863-2895-4441-8015-2e39f0ef6797 admin admin] PATH_INFO: `/v3/projects/a4af24447c7c4b0fb643903b95eed6d1/users/dab4f9eeeef14b3cba346c45b973adab/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.409049 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:39.427204 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e799863-2895-4441-8015-2e39f0ef6797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4af24447c7c4b0fb643903b95eed6d1, user_id=dab4f9eeeef14b3cba346c45b973adab, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:39.430077 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e799863-2895-4441-8015-2e39f0ef6797 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:39.446417 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1306/2678] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:39 2022] PUT /identity/v3/projects/a4af24447c7c4b0fb643903b95eed6d1/users/dab4f9eeeef14b3cba346c45b973adab/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:39.457746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bff0bf-6930-4de0-baeb-4a3d84b1b118 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.458037 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bff0bf-6930-4de0-baeb-4a3d84b1b118 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.458319 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bff0bf-6930-4de0-baeb-4a3d84b1b118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.490956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-f8bff0bf-6930-4de0-baeb-4a3d84b1b118 None None] MFA Rules not processed for user `dab4f9eeeef14b3cba346c45b973adab`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:39.492300 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f8bff0bf-6930-4de0-baeb-4a3d84b1b118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:39.564388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1373/2679] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:39.565800 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e799863-2895-4441-8015-2e39f0ef6797 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:39.633140 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e799863-2895-4441-8015-2e39f0ef6797 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:39.637913 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e1413-24cc-4873-8dce-79138b0d77db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:39.638586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e1413-24cc-4873-8dce-79138b0d77db admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.638924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e1413-24cc-4873-8dce-79138b0d77db admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.639252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e1413-24cc-4873-8dce-79138b0d77db admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.640021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e1413-24cc-4873-8dce-79138b0d77db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:39.642106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e1413-24cc-4873-8dce-79138b0d77db admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:39.678468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1307/2680] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:39 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:39.679925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bff0bf-6930-4de0-baeb-4a3d84b1b118 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:39.692177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bff0bf-6930-4de0-baeb-4a3d84b1b118 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:39.755639 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5b951-553e-4f47-9b6a-0281b37135dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:39.756327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5b951-553e-4f47-9b6a-0281b37135dd admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.756614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5b951-553e-4f47-9b6a-0281b37135dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.756905 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5b951-553e-4f47-9b6a-0281b37135dd admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.757632 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5b951-553e-4f47-9b6a-0281b37135dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:39.760446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5b951-553e-4f47-9b6a-0281b37135dd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:39.785502 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1374/2681] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:39 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:39.789536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e1413-24cc-4873-8dce-79138b0d77db admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:39.800546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e1413-24cc-4873-8dce-79138b0d77db admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:39.805586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839bac4-bab6-4a5f-8cd3-c4c14b4a71ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:39.806266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839bac4-bab6-4a5f-8cd3-c4c14b4a71ff admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.806544 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839bac4-bab6-4a5f-8cd3-c4c14b4a71ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.806819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839bac4-bab6-4a5f-8cd3-c4c14b4a71ff admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.807763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839bac4-bab6-4a5f-8cd3-c4c14b4a71ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:39.810059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839bac4-bab6-4a5f-8cd3-c4c14b4a71ff admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:39.822481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1308/2682] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:39.825898 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5b951-553e-4f47-9b6a-0281b37135dd admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:39.836345 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5b951-553e-4f47-9b6a-0281b37135dd admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:39.841063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040c05e-38c1-460e-a312-174cc414d4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:39.841778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040c05e-38c1-460e-a312-174cc414d4fe admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.842068 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040c05e-38c1-460e-a312-174cc414d4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.842342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040c05e-38c1-460e-a312-174cc414d4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.843071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040c05e-38c1-460e-a312-174cc414d4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:39.844641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040c05e-38c1-460e-a312-174cc414d4fe admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:39.854835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1375/2683] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:00:39.859898 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839bac4-bab6-4a5f-8cd3-c4c14b4a71ff admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:39.871139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839bac4-bab6-4a5f-8cd3-c4c14b4a71ff admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:39.876907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b741ac-a35d-41e5-8b67-ea57595b645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:39.877833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b741ac-a35d-41e5-8b67-ea57595b645e admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.878123 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b741ac-a35d-41e5-8b67-ea57595b645e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.878406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b741ac-a35d-41e5-8b67-ea57595b645e admin admin] PATH_INFO: `/v3/projects/0665d74b566345f5bf8e5939f4295202/users/fe95a40b84454f9f9c9d11d6a74dd421/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.893938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b741ac-a35d-41e5-8b67-ea57595b645e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0665d74b566345f5bf8e5939f4295202, user_id=fe95a40b84454f9f9c9d11d6a74dd421, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:39.896590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b741ac-a35d-41e5-8b67-ea57595b645e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:39.913547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1309/2684] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:39 2022] PUT /identity/v3/projects/0665d74b566345f5bf8e5939f4295202/users/fe95a40b84454f9f9c9d11d6a74dd421/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:39.925524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c6865-7ec3-4ff0-abd8-db0daf5a022f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:39.925826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c6865-7ec3-4ff0-abd8-db0daf5a022f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:39.926110 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c6865-7ec3-4ff0-abd8-db0daf5a022f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:39.953574 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-605c6865-7ec3-4ff0-abd8-db0daf5a022f None None] MFA Rules not processed for user `fe95a40b84454f9f9c9d11d6a74dd421`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:39.954742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-605c6865-7ec3-4ff0-abd8-db0daf5a022f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:39.999555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:40.023907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1376/2685] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:40.029082 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b741ac-a35d-41e5-8b67-ea57595b645e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:40.092318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b741ac-a35d-41e5-8b67-ea57595b645e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:40.098282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b444184-5032-4518-945f-4e5279cf4f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:40.099026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b444184-5032-4518-945f-4e5279cf4f16 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:40.099337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b444184-5032-4518-945f-4e5279cf4f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:40.099637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b444184-5032-4518-945f-4e5279cf4f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:40.100438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b444184-5032-4518-945f-4e5279cf4f16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:40.102385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b444184-5032-4518-945f-4e5279cf4f16 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:40.134493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1310/2686] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:40 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:40.139598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c6865-7ec3-4ff0-abd8-db0daf5a022f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:40.155658 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c6865-7ec3-4ff0-abd8-db0daf5a022f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:40.214835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63361946-8080-41c7-ba56-cd6fbfc44bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:40.215217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63361946-8080-41c7-ba56-cd6fbfc44bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:40.215511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63361946-8080-41c7-ba56-cd6fbfc44bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:40.216281 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63361946-8080-41c7-ba56-cd6fbfc44bb1 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:40.216965 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63361946-8080-41c7-ba56-cd6fbfc44bb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:40.218904 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63361946-8080-41c7-ba56-cd6fbfc44bb1 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:40.240423 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1377/2687] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:40 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:40.243391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b444184-5032-4518-945f-4e5279cf4f16 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:40.253669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b444184-5032-4518-945f-4e5279cf4f16 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:40.258333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d1e76-b72f-4893-90fb-c9518bee5e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:40.258991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d1e76-b72f-4893-90fb-c9518bee5e30 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:40.259276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d1e76-b72f-4893-90fb-c9518bee5e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:40.259573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d1e76-b72f-4893-90fb-c9518bee5e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:40.260280 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d1e76-b72f-4893-90fb-c9518bee5e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:40.261896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d1e76-b72f-4893-90fb-c9518bee5e30 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:40.272558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1311/2688] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:00:40.276792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63361946-8080-41c7-ba56-cd6fbfc44bb1 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:40.287858 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63361946-8080-41c7-ba56-cd6fbfc44bb1 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:40.292854 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0cedb-6323-451f-abc8-fdbdb1b5087a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:40.293771 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0cedb-6323-451f-abc8-fdbdb1b5087a admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:40.294720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0cedb-6323-451f-abc8-fdbdb1b5087a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:40.294997 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0cedb-6323-451f-abc8-fdbdb1b5087a admin admin] PATH_INFO: `/v3/projects/5cc4c9b7e3fd40089c5e028afa1af4f6/users/97f0c4e8888b4a2980da6b07b9888883/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:40.309626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0cedb-6323-451f-abc8-fdbdb1b5087a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc4c9b7e3fd40089c5e028afa1af4f6, user_id=97f0c4e8888b4a2980da6b07b9888883, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:40.312365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0cedb-6323-451f-abc8-fdbdb1b5087a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:40.325682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1378/2689] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:40 2022] PUT /identity/v3/projects/5cc4c9b7e3fd40089c5e028afa1af4f6/users/97f0c4e8888b4a2980da6b07b9888883/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:40.329605 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d1e76-b72f-4893-90fb-c9518bee5e30 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:40.339562 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d1e76-b72f-4893-90fb-c9518bee5e30 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:40.343716 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:40.401283 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2e038-09ac-4064-b905-7caa1b09ee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:40.401980 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2e038-09ac-4064-b905-7caa1b09ee02 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:40.402262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2e038-09ac-4064-b905-7caa1b09ee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:40.402565 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2e038-09ac-4064-b905-7caa1b09ee02 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:40.403932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2e038-09ac-4064-b905-7caa1b09ee02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:40.405634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2e038-09ac-4064-b905-7caa1b09ee02 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:40.419340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1312/2690] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:40.420406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0cedb-6323-451f-abc8-fdbdb1b5087a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:40.431260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0cedb-6323-451f-abc8-fdbdb1b5087a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:40.436444 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1649aa9-94dc-4571-aa37-dae3ea05a148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:40.437335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649aa9-94dc-4571-aa37-dae3ea05a148 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:40.439051 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649aa9-94dc-4571-aa37-dae3ea05a148 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:40.439051 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1649aa9-94dc-4571-aa37-dae3ea05a148 admin admin] PATH_INFO: `/v3/projects/5cc4c9b7e3fd40089c5e028afa1af4f6/users/97f0c4e8888b4a2980da6b07b9888883/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:40.440321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1649aa9-94dc-4571-aa37-dae3ea05a148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc4c9b7e3fd40089c5e028afa1af4f6, user_id=97f0c4e8888b4a2980da6b07b9888883, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:40.442643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1649aa9-94dc-4571-aa37-dae3ea05a148 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:40.463713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1379/2691] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:40 2022] PUT /identity/v3/projects/5cc4c9b7e3fd40089c5e028afa1af4f6/users/97f0c4e8888b4a2980da6b07b9888883/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:40.478754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e525b6-6603-4bf0-94a5-2cfb62861406 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:40.479040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e525b6-6603-4bf0-94a5-2cfb62861406 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:40.479316 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e525b6-6603-4bf0-94a5-2cfb62861406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:40.517819 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-78e525b6-6603-4bf0-94a5-2cfb62861406 None None] MFA Rules not processed for user `97f0c4e8888b4a2980da6b07b9888883`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:40.519227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-78e525b6-6603-4bf0-94a5-2cfb62861406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:40.593440 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1313/2692] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:40.604931 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1649aa9-94dc-4571-aa37-dae3ea05a148 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:40.607165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a1649aa9-94dc-4571-aa37-dae3ea05a148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:40.663422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1649aa9-94dc-4571-aa37-dae3ea05a148 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:40.667129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d2729-5d8e-490c-a66f-e6787c949058 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:40.667778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d2729-5d8e-490c-a66f-e6787c949058 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:40.668059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d2729-5d8e-490c-a66f-e6787c949058 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:40.668339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d2729-5d8e-490c-a66f-e6787c949058 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:40.677996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d2729-5d8e-490c-a66f-e6787c949058 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:40.679412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d2729-5d8e-490c-a66f-e6787c949058 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:40.688861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1380/2693] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:40.729973 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e525b6-6603-4bf0-94a5-2cfb62861406 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:40.731975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-78e525b6-6603-4bf0-94a5-2cfb62861406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:40.743842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e525b6-6603-4bf0-94a5-2cfb62861406 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:40.747569 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edbed3-6541-45be-a88f-eb77acf5248e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:40.748207 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edbed3-6541-45be-a88f-eb77acf5248e service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:40.748487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edbed3-6541-45be-a88f-eb77acf5248e service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:40.748762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edbed3-6541-45be-a88f-eb77acf5248e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:40.805300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edbed3-6541-45be-a88f-eb77acf5248e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:40.806789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edbed3-6541-45be-a88f-eb77acf5248e service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:40.815828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1314/2694] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:49.800611 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d2729-5d8e-490c-a66f-e6787c949058 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:49.803413 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-ef1d2729-5d8e-490c-a66f-e6787c949058 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:49.867864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:49.961910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d2729-5d8e-490c-a66f-e6787c949058 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:49.980285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718a54f-c90d-4ada-a252-f36c04dec945 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:49.982535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718a54f-c90d-4ada-a252-f36c04dec945 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:49.982854 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718a54f-c90d-4ada-a252-f36c04dec945 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:49.983142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718a54f-c90d-4ada-a252-f36c04dec945 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:49.988644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c718a54f-c90d-4ada-a252-f36c04dec945 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:50.025720 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718a54f-c90d-4ada-a252-f36c04dec945 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:50.033316 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718a54f-c90d-4ada-a252-f36c04dec945 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:50.070510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1381/2695] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:50.185740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edbed3-6541-45be-a88f-eb77acf5248e service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:50.225473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:50.272823 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edbed3-6541-45be-a88f-eb77acf5248e service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:50.290402 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:50.291309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:50.291890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:50.291890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] PATH_INFO: `/v3/users/dab4f9eeeef14b3cba346c45b973adab` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:50.293144 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dab4f9eeeef14b3cba346c45b973adab)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:50.295885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:50.379356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dab4f9eeeef14b3cba346c45b973adab'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:50.412342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dab4f9eeeef14b3cba346c45b973adab'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:50.422608 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dab4f9eeeef14b3cba346c45b973adab'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:50.452942 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dab4f9eeeef14b3cba346c45b973adab'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:50.454226 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dab4f9eeeef14b3cba346c45b973adab'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:50.467991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1315/2696] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:50 2022] DELETE /identity/v3/users/dab4f9eeeef14b3cba346c45b973adab => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:50.473149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec62ada-ba21-4703-99df-1bec0d9f454c None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:50.473686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec62ada-ba21-4703-99df-1bec0d9f454c None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:50.474006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec62ada-ba21-4703-99df-1bec0d9f454c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:50.493893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-0ec62ada-ba21-4703-99df-1bec0d9f454c None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:50.495107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0ec62ada-ba21-4703-99df-1bec0d9f454c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:50.571438 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1382/2697] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:50.586817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:50.588376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:50.671852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb55b1-7ea3-4e8e-a8b4-03b7f67172a8 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:50.678143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631e50-1326-4a28-847c-007e3c638a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:50.679021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631e50-1326-4a28-847c-007e3c638a86 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:50.679374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631e50-1326-4a28-847c-007e3c638a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:50.679728 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631e50-1326-4a28-847c-007e3c638a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:50.692635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d631e50-1326-4a28-847c-007e3c638a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:50.694259 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d631e50-1326-4a28-847c-007e3c638a86 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:50.706653 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1316/2698] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:51.493594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec62ada-ba21-4703-99df-1bec0d9f454c None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:51.495191 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0ec62ada-ba21-4703-99df-1bec0d9f454c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:51.521028 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec62ada-ba21-4703-99df-1bec0d9f454c None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:51.535828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:51.540253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:51.540253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:51.542080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] PATH_INFO: `/v3/projects/a4af24447c7c4b0fb643903b95eed6d1` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:51.545341 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4af24447c7c4b0fb643903b95eed6d1)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:51.549282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:51.664075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4af24447c7c4b0fb643903b95eed6d1', 'request_id': 'req-f98a525b-0f18-4fcf-9a80-416d809ea4e4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:51.680815 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4af24447c7c4b0fb643903b95eed6d1', 'request_id': 'req-f98a525b-0f18-4fcf-9a80-416d809ea4e4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:51.692419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4af24447c7c4b0fb643903b95eed6d1', 'request_id': 'req-f98a525b-0f18-4fcf-9a80-416d809ea4e4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:51.715988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] The token cache is being invalidate because project a4af24447c7c4b0fb643903b95eed6d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:51.715988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:51.717663 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:51.720139 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1383/2699] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:51 2022] DELETE /identity/v3/projects/a4af24447c7c4b0fb643903b95eed6d1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:51.724215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631e50-1326-4a28-847c-007e3c638a86 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:51.742373 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631e50-1326-4a28-847c-007e3c638a86 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:51.858330 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:51.887487 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:51.892189 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:51.892855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:51.893212 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] PATH_INFO: `/v3/users/fe95a40b84454f9f9c9d11d6a74dd421` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:51.895323 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe95a40b84454f9f9c9d11d6a74dd421)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:51.898021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:52.122118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe95a40b84454f9f9c9d11d6a74dd421'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:52.133683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe95a40b84454f9f9c9d11d6a74dd421'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:52.141977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe95a40b84454f9f9c9d11d6a74dd421'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:52.155267 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe95a40b84454f9f9c9d11d6a74dd421'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:52.156116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe95a40b84454f9f9c9d11d6a74dd421'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:52.162738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1317/2700] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:51 2022] DELETE /identity/v3/users/fe95a40b84454f9f9c9d11d6a74dd421 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:52.603153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:52.604698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:52.620249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a525b-0f18-4fcf-9a80-416d809ea4e4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:52.708433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:52.739668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:52.740697 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:52.740988 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:52.741275 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] PATH_INFO: `/v3/projects/0665d74b566345f5bf8e5939f4295202` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:52.742699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0665d74b566345f5bf8e5939f4295202)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:52.744656 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:52.821278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0665d74b566345f5bf8e5939f4295202', 'request_id': 'req-4109f9e2-88a3-497e-99c0-d267d1027092'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:52.834546 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0665d74b566345f5bf8e5939f4295202', 'request_id': 'req-4109f9e2-88a3-497e-99c0-d267d1027092'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:52.843834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0665d74b566345f5bf8e5939f4295202', 'request_id': 'req-4109f9e2-88a3-497e-99c0-d267d1027092'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:52.863392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] The token cache is being invalidate because project 0665d74b566345f5bf8e5939f4295202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:52.863696 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:52.864375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:52.869283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1384/2701] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:52 2022] DELETE /identity/v3/projects/0665d74b566345f5bf8e5939f4295202 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:52.874275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:52.875710 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:52.894219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd798902-3cf0-42c9-9486-2180bd2d0303 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:53.039121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039871e-b993-4f87-bf7b-0ff32a2b1f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:53.040597 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039871e-b993-4f87-bf7b-0ff32a2b1f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:53.040884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039871e-b993-4f87-bf7b-0ff32a2b1f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:53.041294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039871e-b993-4f87-bf7b-0ff32a2b1f2d admin admin] PATH_INFO: `/v3/users/97f0c4e8888b4a2980da6b07b9888883` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:53.043198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039871e-b993-4f87-bf7b-0ff32a2b1f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f0c4e8888b4a2980da6b07b9888883)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:53.046491 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039871e-b993-4f87-bf7b-0ff32a2b1f2d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:53.138667 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e039871e-b993-4f87-bf7b-0ff32a2b1f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f0c4e8888b4a2980da6b07b9888883'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:53.151738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e039871e-b993-4f87-bf7b-0ff32a2b1f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f0c4e8888b4a2980da6b07b9888883'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:53.161278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e039871e-b993-4f87-bf7b-0ff32a2b1f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f0c4e8888b4a2980da6b07b9888883'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:53.168172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:53.179195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e039871e-b993-4f87-bf7b-0ff32a2b1f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f0c4e8888b4a2980da6b07b9888883'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:53.180170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-e039871e-b993-4f87-bf7b-0ff32a2b1f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f0c4e8888b4a2980da6b07b9888883'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:53.187830 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1318/2702] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:00:52 2022] DELETE /identity/v3/users/97f0c4e8888b4a2980da6b07b9888883 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:53.507464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:53.509216 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:53.528677 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109f9e2-88a3-497e-99c0-d267d1027092 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:53.588944 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:53.589759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:53.590045 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:53.590326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] PATH_INFO: `/v3/projects/5cc4c9b7e3fd40089c5e028afa1af4f6` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:53.591526 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc4c9b7e3fd40089c5e028afa1af4f6)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:53.593486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:53.650782 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc4c9b7e3fd40089c5e028afa1af4f6', 'request_id': 'req-38644d9b-9e56-44f8-9a55-2b8da32736a8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:53.658493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc4c9b7e3fd40089c5e028afa1af4f6', 'request_id': 'req-38644d9b-9e56-44f8-9a55-2b8da32736a8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:53.664475 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc4c9b7e3fd40089c5e028afa1af4f6', 'request_id': 'req-38644d9b-9e56-44f8-9a55-2b8da32736a8'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:53.676370 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] The token cache is being invalidate because project 5cc4c9b7e3fd40089c5e028afa1af4f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:53.676664 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:53.677327 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-38644d9b-9e56-44f8-9a55-2b8da32736a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:53.678891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1385/2703] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:53 2022] DELETE /identity/v3/projects/5cc4c9b7e3fd40089c5e028afa1af4f6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:53.685318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50c246-083f-4377-8c56-1caf2a03dd98 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:53.685748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50c246-083f-4377-8c56-1caf2a03dd98 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:53.686060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50c246-083f-4377-8c56-1caf2a03dd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:53.704771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ad50c246-083f-4377-8c56-1caf2a03dd98 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:53.706036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ad50c246-083f-4377-8c56-1caf2a03dd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:53.768279 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1319/2704] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:53.777776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1161d-c1ab-462b-8db6-7bb394f135c1 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:53.778060 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1161d-c1ab-462b-8db6-7bb394f135c1 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:53.778335 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1161d-c1ab-462b-8db6-7bb394f135c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:53.790879 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:53.800464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-08c1161d-c1ab-462b-8db6-7bb394f135c1 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:53.802647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-08c1161d-c1ab-462b-8db6-7bb394f135c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:53.813043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1386/2705] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:53.817634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50c246-083f-4377-8c56-1caf2a03dd98 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:53.828066 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50c246-083f-4377-8c56-1caf2a03dd98 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:53.832677 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3de0e-0987-4392-98a9-87f8d6de625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:53.833326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3de0e-0987-4392-98a9-87f8d6de625e admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:53.833630 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3de0e-0987-4392-98a9-87f8d6de625e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:53.833906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3de0e-0987-4392-98a9-87f8d6de625e admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:53.834621 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3de0e-0987-4392-98a9-87f8d6de625e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:00:53.834925 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3de0e-0987-4392-98a9-87f8d6de625e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:53.836507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3de0e-0987-4392-98a9-87f8d6de625e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:53.849185 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1320/2706] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:00:53.853570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1161d-c1ab-462b-8db6-7bb394f135c1 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:53.863948 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1161d-c1ab-462b-8db6-7bb394f135c1 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:53.868732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9c82e-23b8-4349-9280-6275a8486cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:53.869421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9c82e-23b8-4349-9280-6275a8486cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:53.869710 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9c82e-23b8-4349-9280-6275a8486cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:53.869993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9c82e-23b8-4349-9280-6275a8486cd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:53.870692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9c82e-23b8-4349-9280-6275a8486cd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:53.873189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9c82e-23b8-4349-9280-6275a8486cd0 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:53.901433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1387/2707] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:53 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:53.906146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3de0e-0987-4392-98a9-87f8d6de625e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:53.916194 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3de0e-0987-4392-98a9-87f8d6de625e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:53.990083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d802a6-b259-46e4-a07d-6b7ee0406912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:53.990762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d802a6-b259-46e4-a07d-6b7ee0406912 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:53.991044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d802a6-b259-46e4-a07d-6b7ee0406912 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:53.991319 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d802a6-b259-46e4-a07d-6b7ee0406912 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:53.992020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d802a6-b259-46e4-a07d-6b7ee0406912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:53.994020 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d802a6-b259-46e4-a07d-6b7ee0406912 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:54.015906 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1321/2708] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:53 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:54.020189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9c82e-23b8-4349-9280-6275a8486cd0 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:54.033279 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9c82e-23b8-4349-9280-6275a8486cd0 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:54.038651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f9b41-d212-493f-8974-e2d5fba06eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:54.039504 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f9b41-d212-493f-8974-e2d5fba06eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.039951 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f9b41-d212-493f-8974-e2d5fba06eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.040329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f9b41-d212-493f-8974-e2d5fba06eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.041378 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f9b41-d212-493f-8974-e2d5fba06eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:54.043103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f9b41-d212-493f-8974-e2d5fba06eb1 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:54.055432 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1388/2709] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:54.059408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d802a6-b259-46e4-a07d-6b7ee0406912 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:54.071430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d802a6-b259-46e4-a07d-6b7ee0406912 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:54.076924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51660d5-83de-4d5d-9885-b643864c2ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:54.077619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51660d5-83de-4d5d-9885-b643864c2ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.077905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51660d5-83de-4d5d-9885-b643864c2ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.078178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51660d5-83de-4d5d-9885-b643864c2ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.078909 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51660d5-83de-4d5d-9885-b643864c2ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:54.080493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51660d5-83de-4d5d-9885-b643864c2ec1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:54.092639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1322/2710] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:54.097291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f9b41-d212-493f-8974-e2d5fba06eb1 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:54.107456 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f9b41-d212-493f-8974-e2d5fba06eb1 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:54.112156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3659cee-43c0-4674-9342-c880f4df7865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:54.113026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3659cee-43c0-4674-9342-c880f4df7865 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.113309 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3659cee-43c0-4674-9342-c880f4df7865 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.113615 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3659cee-43c0-4674-9342-c880f4df7865 admin admin] PATH_INFO: `/v3/projects/92e8502d8bed40458df35e622556bdb0/users/5001397b955c4e51937b71538a559c07/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.129217 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3659cee-43c0-4674-9342-c880f4df7865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e8502d8bed40458df35e622556bdb0, user_id=5001397b955c4e51937b71538a559c07, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:54.131912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3659cee-43c0-4674-9342-c880f4df7865 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:54.147337 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1389/2711] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:54 2022] PUT /identity/v3/projects/92e8502d8bed40458df35e622556bdb0/users/5001397b955c4e51937b71538a559c07/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:00:54.159993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ebbdf-c565-4b06-a9e1-1a8db7ad80ac None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.160284 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ebbdf-c565-4b06-a9e1-1a8db7ad80ac None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.160576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ebbdf-c565-4b06-a9e1-1a8db7ad80ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.189322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-b83ebbdf-c565-4b06-a9e1-1a8db7ad80ac None None] MFA Rules not processed for user `5001397b955c4e51937b71538a559c07`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:54.190639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b83ebbdf-c565-4b06-a9e1-1a8db7ad80ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:54.235993 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:54.260206 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1323/2712] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:54.264694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3659cee-43c0-4674-9342-c880f4df7865 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:54.330357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3659cee-43c0-4674-9342-c880f4df7865 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:54.335330 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60560d-ef15-4f3b-ab76-e20c80b593eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:54.335992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60560d-ef15-4f3b-ab76-e20c80b593eb admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.336274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60560d-ef15-4f3b-ab76-e20c80b593eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.336579 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60560d-ef15-4f3b-ab76-e20c80b593eb admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.337247 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60560d-ef15-4f3b-ab76-e20c80b593eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:54.339162 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60560d-ef15-4f3b-ab76-e20c80b593eb admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:54.370921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1390/2713] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:00:54 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:54.375054 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ebbdf-c565-4b06-a9e1-1a8db7ad80ac None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:54.391981 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ebbdf-c565-4b06-a9e1-1a8db7ad80ac None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:54.453855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ec4d8-a7a3-4425-86a6-86a97975a243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:54.454507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ec4d8-a7a3-4425-86a6-86a97975a243 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.454930 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ec4d8-a7a3-4425-86a6-86a97975a243 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.455073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ec4d8-a7a3-4425-86a6-86a97975a243 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.455823 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ec4d8-a7a3-4425-86a6-86a97975a243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:54.457801 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ec4d8-a7a3-4425-86a6-86a97975a243 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:54.485833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1324/2714] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:00:54 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:00:54.489871 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60560d-ef15-4f3b-ab76-e20c80b593eb admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:54.502880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60560d-ef15-4f3b-ab76-e20c80b593eb admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:54.507740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d245b-d5cc-4c7f-845e-3b0f73563ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:54.508416 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d245b-d5cc-4c7f-845e-3b0f73563ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.508713 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d245b-d5cc-4c7f-845e-3b0f73563ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.508996 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d245b-d5cc-4c7f-845e-3b0f73563ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.509768 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d245b-d5cc-4c7f-845e-3b0f73563ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:54.511366 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d245b-d5cc-4c7f-845e-3b0f73563ef2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:54.523527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1391/2715] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:54.527720 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ec4d8-a7a3-4425-86a6-86a97975a243 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:54.539471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ec4d8-a7a3-4425-86a6-86a97975a243 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:54.545080 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9bdc2-1a3e-41bf-85d9-2578a5e8ddb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:54.546044 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9bdc2-1a3e-41bf-85d9-2578a5e8ddb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.546353 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9bdc2-1a3e-41bf-85d9-2578a5e8ddb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.546634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9bdc2-1a3e-41bf-85d9-2578a5e8ddb8 admin admin] PATH_INFO: `/v3/projects/a292f6648e3c4f6abef5aa1cf5fbb389/users/61184d370fcc4888a4181642a2bedc2f/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.562675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9bdc2-1a3e-41bf-85d9-2578a5e8ddb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a292f6648e3c4f6abef5aa1cf5fbb389, user_id=61184d370fcc4888a4181642a2bedc2f, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:54.565298 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9bdc2-1a3e-41bf-85d9-2578a5e8ddb8 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:54.581667 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1325/2716] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:54 2022] PUT /identity/v3/projects/a292f6648e3c4f6abef5aa1cf5fbb389/users/61184d370fcc4888a4181642a2bedc2f/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:54.585763 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d245b-d5cc-4c7f-845e-3b0f73563ef2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:54.596801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d245b-d5cc-4c7f-845e-3b0f73563ef2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:54.655409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a923f-1b22-4d64-bc1b-3e206c556acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:54.656112 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a923f-1b22-4d64-bc1b-3e206c556acf admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.656260 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a923f-1b22-4d64-bc1b-3e206c556acf admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.656534 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a923f-1b22-4d64-bc1b-3e206c556acf admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.662717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a923f-1b22-4d64-bc1b-3e206c556acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:54.662717 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a923f-1b22-4d64-bc1b-3e206c556acf admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:54.674737 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1392/2717] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:00:54.679067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9bdc2-1a3e-41bf-85d9-2578a5e8ddb8 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:54.689595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9bdc2-1a3e-41bf-85d9-2578a5e8ddb8 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:54.694255 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111176c-47ab-4b9a-a243-10140565b89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:54.695137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111176c-47ab-4b9a-a243-10140565b89c admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.695425 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111176c-47ab-4b9a-a243-10140565b89c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.695707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111176c-47ab-4b9a-a243-10140565b89c admin admin] PATH_INFO: `/v3/projects/a292f6648e3c4f6abef5aa1cf5fbb389/users/61184d370fcc4888a4181642a2bedc2f/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.697745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111176c-47ab-4b9a-a243-10140565b89c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a292f6648e3c4f6abef5aa1cf5fbb389, user_id=61184d370fcc4888a4181642a2bedc2f, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:54.700847 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111176c-47ab-4b9a-a243-10140565b89c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:54.711021 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1326/2718] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:00:54 2022] PUT /identity/v3/projects/a292f6648e3c4f6abef5aa1cf5fbb389/users/61184d370fcc4888a4181642a2bedc2f/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:54.722259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fd942-0fd9-444a-8fa2-f9820b71c02b None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.722541 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fd942-0fd9-444a-8fa2-f9820b71c02b None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.722819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fd942-0fd9-444a-8fa2-f9820b71c02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.746620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:54.749909 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-3c6fd942-0fd9-444a-8fa2-f9820b71c02b None None] MFA Rules not processed for user `61184d370fcc4888a4181642a2bedc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:54.751551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3c6fd942-0fd9-444a-8fa2-f9820b71c02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:54.818971 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1393/2719] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:00:54 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:54.831304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111176c-47ab-4b9a-a243-10140565b89c admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:54.833052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a111176c-47ab-4b9a-a243-10140565b89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:54.891637 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111176c-47ab-4b9a-a243-10140565b89c admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:54.895472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccde18-ad4c-4f9a-b222-18e6b34cedd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:54.896116 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccde18-ad4c-4f9a-b222-18e6b34cedd3 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:54.896406 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccde18-ad4c-4f9a-b222-18e6b34cedd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:54.896689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccde18-ad4c-4f9a-b222-18e6b34cedd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:54.905861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccde18-ad4c-4f9a-b222-18e6b34cedd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:54.907223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccde18-ad4c-4f9a-b222-18e6b34cedd3 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:54.916779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1327/2720] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:54 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:54.996244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fd942-0fd9-444a-8fa2-f9820b71c02b None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:55.006975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fd942-0fd9-444a-8fa2-f9820b71c02b None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:55.010762 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58044576-6e7b-4d64-af95-f52d9d0c89ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:55.011835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58044576-6e7b-4d64-af95-f52d9d0c89ea service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:55.012157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58044576-6e7b-4d64-af95-f52d9d0c89ea service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:55.012436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58044576-6e7b-4d64-af95-f52d9d0c89ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:55.071799 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58044576-6e7b-4d64-af95-f52d9d0c89ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:55.073298 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58044576-6e7b-4d64-af95-f52d9d0c89ea service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:55.083303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1394/2721] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:03.062665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccde18-ad4c-4f9a-b222-18e6b34cedd3 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:03.104593 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:03.130443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccde18-ad4c-4f9a-b222-18e6b34cedd3 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:03.135119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:03.135928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:03.136208 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:03.136483 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] PATH_INFO: `/v3/users/5001397b955c4e51937b71538a559c07` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:03.137696 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5001397b955c4e51937b71538a559c07)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:03.139595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:03.199779 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5001397b955c4e51937b71538a559c07'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:03.212576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5001397b955c4e51937b71538a559c07'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:03.224230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5001397b955c4e51937b71538a559c07'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:03.241955 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5001397b955c4e51937b71538a559c07'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:03.243410 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5001397b955c4e51937b71538a559c07'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:03.255459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1328/2722] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:01:03 2022] DELETE /identity/v3/users/5001397b955c4e51937b71538a559c07 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:01:03.260156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e1e6c-4e23-40b5-aa7b-83252aade819 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:03.260686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e1e6c-4e23-40b5-aa7b-83252aade819 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:03.260686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e1e6c-4e23-40b5-aa7b-83252aade819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:03.280834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-9e7e1e6c-4e23-40b5-aa7b-83252aade819 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:03.282128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9e7e1e6c-4e23-40b5-aa7b-83252aade819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:03.332170 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:03.333763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:03.385643 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1395/2723] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:03.402428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e1e6c-4e23-40b5-aa7b-83252aade819 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:03.404975 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9e7e1e6c-4e23-40b5-aa7b-83252aade819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:03.413551 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e92ac-fc3b-404a-be67-6f14c50a202d admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:03.431535 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:03.433652 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:03.434733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:03.435196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] PATH_INFO: `/v3/users/792d034bc1fd424e9c00356c2b6eb2f6` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:03.436763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=792d034bc1fd424e9c00356c2b6eb2f6)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:03.438982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:03.487647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:03.516961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '792d034bc1fd424e9c00356c2b6eb2f6'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:03.517245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e1e6c-4e23-40b5-aa7b-83252aade819 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:03.524806 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb837d34-d446-4eb6-9532-81804f0555f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:03.526388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb837d34-d446-4eb6-9532-81804f0555f6 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:03.527024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb837d34-d446-4eb6-9532-81804f0555f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:03.527524 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb837d34-d446-4eb6-9532-81804f0555f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:03.529011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '792d034bc1fd424e9c00356c2b6eb2f6'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:03.536093 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:03.546235 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '792d034bc1fd424e9c00356c2b6eb2f6'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:03.548106 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb837d34-d446-4eb6-9532-81804f0555f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:03.551070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb837d34-d446-4eb6-9532-81804f0555f6 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:03.568742 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '792d034bc1fd424e9c00356c2b6eb2f6'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:03.569664 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '792d034bc1fd424e9c00356c2b6eb2f6'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:03.573170 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1396/2724] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:01:03.577776 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1329/2725] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:01:03 2022] DELETE /identity/v3/users/792d034bc1fd424e9c00356c2b6eb2f6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:01:03.584525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aba326-4680-479c-9b8b-3c2c79dcd666 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:03.585313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aba326-4680-479c-9b8b-3c2c79dcd666 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:03.585888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aba326-4680-479c-9b8b-3c2c79dcd666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:03.605828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-18aba326-4680-479c-9b8b-3c2c79dcd666 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:03.607896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-18aba326-4680-479c-9b8b-3c2c79dcd666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:03.701095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1397/2726] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:03.711456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:03.714139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:03.813663 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335c8b0-5cf3-41bd-ab4b-d27b7ae68365 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:03.820547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910f0f3-4188-4f87-96df-c02bfda39336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:03.821969 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910f0f3-4188-4f87-96df-c02bfda39336 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:03.822550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910f0f3-4188-4f87-96df-c02bfda39336 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:03.823095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910f0f3-4188-4f87-96df-c02bfda39336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:03.842192 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910f0f3-4188-4f87-96df-c02bfda39336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:03.843349 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910f0f3-4188-4f87-96df-c02bfda39336 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:03.855213 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1330/2727] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:03.885047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aba326-4680-479c-9b8b-3c2c79dcd666 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:03.890150 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-18aba326-4680-479c-9b8b-3c2c79dcd666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:04.062406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aba326-4680-479c-9b8b-3c2c79dcd666 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:04.071218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824daf4e-4b11-4ca2-8a1a-3e031c34e3ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:04.072244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824daf4e-4b11-4ca2-8a1a-3e031c34e3ca service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:04.072523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824daf4e-4b11-4ca2-8a1a-3e031c34e3ca service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:04.072759 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824daf4e-4b11-4ca2-8a1a-3e031c34e3ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:04.078561 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-824daf4e-4b11-4ca2-8a1a-3e031c34e3ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:04.095540 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824daf4e-4b11-4ca2-8a1a-3e031c34e3ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:04.097493 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824daf4e-4b11-4ca2-8a1a-3e031c34e3ca service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:04.111206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1398/2728] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:01:04.190729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910f0f3-4188-4f87-96df-c02bfda39336 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:04.193996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c910f0f3-4188-4f87-96df-c02bfda39336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:04.225288 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910f0f3-4188-4f87-96df-c02bfda39336 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:04.232684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:04.234975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:04.235269 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:04.235548 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] PATH_INFO: `/v3/projects/92e8502d8bed40458df35e622556bdb0` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:04.237097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e8502d8bed40458df35e622556bdb0)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:04.239113 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:04.295549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824daf4e-4b11-4ca2-8a1a-3e031c34e3ca service nova] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:04.296750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-824daf4e-4b11-4ca2-8a1a-3e031c34e3ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:04.305376 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e8502d8bed40458df35e622556bdb0', 'request_id': 'req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.316309 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e8502d8bed40458df35e622556bdb0', 'request_id': 'req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.319758 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824daf4e-4b11-4ca2-8a1a-3e031c34e3ca service nova] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:04.328977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e8502d8bed40458df35e622556bdb0', 'request_id': 'req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.359748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] The token cache is being invalidate because project 92e8502d8bed40458df35e622556bdb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:01:04.360040 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.363227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.368880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1331/2729] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:01:04 2022] DELETE /identity/v3/projects/92e8502d8bed40458df35e622556bdb0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:04.370693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:04.380965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:04.451168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:04.452189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:04.452527 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:04.452817 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] PATH_INFO: `/v3/projects/94975be780694882b1689446ae402965` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:04.454286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94975be780694882b1689446ae402965)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:04.456293 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:04.470148 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:04.486665 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d1ae4-1b53-49fd-9d51-88a9fcab4083 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:04.492788 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39009b00-7e53-4073-bd59-7dd0d48b26c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:04.493931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009b00-7e53-4073-bd59-7dd0d48b26c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:04.494220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009b00-7e53-4073-bd59-7dd0d48b26c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:04.494498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009b00-7e53-4073-bd59-7dd0d48b26c9 admin admin] PATH_INFO: `/v3/users/61184d370fcc4888a4181642a2bedc2f` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:04.495797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39009b00-7e53-4073-bd59-7dd0d48b26c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61184d370fcc4888a4181642a2bedc2f)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:04.499024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39009b00-7e53-4073-bd59-7dd0d48b26c9 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:04.502171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:04.544851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94975be780694882b1689446ae402965', 'request_id': 'req-7b099326-c87e-410f-8685-1b8bbfa8b098'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.554036 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94975be780694882b1689446ae402965', 'request_id': 'req-7b099326-c87e-410f-8685-1b8bbfa8b098'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.563407 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94975be780694882b1689446ae402965', 'request_id': 'req-7b099326-c87e-410f-8685-1b8bbfa8b098'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.569452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-39009b00-7e53-4073-bd59-7dd0d48b26c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61184d370fcc4888a4181642a2bedc2f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.578520 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-39009b00-7e53-4073-bd59-7dd0d48b26c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61184d370fcc4888a4181642a2bedc2f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.579319 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] The token cache is being invalidate because project 94975be780694882b1689446ae402965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:01:04.579618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.580375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-7b099326-c87e-410f-8685-1b8bbfa8b098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.582733 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1399/2730] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:01:04 2022] DELETE /identity/v3/projects/94975be780694882b1689446ae402965 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:01:04.586768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-39009b00-7e53-4073-bd59-7dd0d48b26c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61184d370fcc4888a4181642a2bedc2f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.589056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37e065-449c-485b-953a-5153998f2d9e None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:04.589347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37e065-449c-485b-953a-5153998f2d9e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:04.589666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37e065-449c-485b-953a-5153998f2d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:04.600239 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-39009b00-7e53-4073-bd59-7dd0d48b26c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61184d370fcc4888a4181642a2bedc2f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.601036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-39009b00-7e53-4073-bd59-7dd0d48b26c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61184d370fcc4888a4181642a2bedc2f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:04.608227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1332/2731] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:01:04 2022] DELETE /identity/v3/users/61184d370fcc4888a4181642a2bedc2f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:04.611323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-3d37e065-449c-485b-953a-5153998f2d9e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:04.611323 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3d37e065-449c-485b-953a-5153998f2d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:04.684290 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1400/2732] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:01:04.693121 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec034b26-9b83-4d22-96fe-f9608851c24e None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:04.693604 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec034b26-9b83-4d22-96fe-f9608851c24e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:04.694092 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec034b26-9b83-4d22-96fe-f9608851c24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:04.716975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-ec034b26-9b83-4d22-96fe-f9608851c24e None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:04.718486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ec034b26-9b83-4d22-96fe-f9608851c24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:04.728475 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1333/2733] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:04.732947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37e065-449c-485b-953a-5153998f2d9e None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:04.746006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37e065-449c-485b-953a-5153998f2d9e None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:04.751740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abcdb3-2154-435c-aebe-ff6205bb63ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:04.752423 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abcdb3-2154-435c-aebe-ff6205bb63ab admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:04.752704 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abcdb3-2154-435c-aebe-ff6205bb63ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:04.752976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abcdb3-2154-435c-aebe-ff6205bb63ab admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:04.753794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abcdb3-2154-435c-aebe-ff6205bb63ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:01:04.754053 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abcdb3-2154-435c-aebe-ff6205bb63ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:04.755756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abcdb3-2154-435c-aebe-ff6205bb63ab admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:04.771943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1401/2734] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 21:01:04.776560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec034b26-9b83-4d22-96fe-f9608851c24e None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:04.790198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec034b26-9b83-4d22-96fe-f9608851c24e None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:04.796994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c860f3b-49c3-430f-b837-b6c4c184fc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:04.797792 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c860f3b-49c3-430f-b837-b6c4c184fc9f admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:04.798062 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c860f3b-49c3-430f-b837-b6c4c184fc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:04.798434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c860f3b-49c3-430f-b837-b6c4c184fc9f admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:04.799231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c860f3b-49c3-430f-b837-b6c4c184fc9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:04.801457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c860f3b-49c3-430f-b837-b6c4c184fc9f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:04.828271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1334/2735] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:01:04 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:04.832775 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abcdb3-2154-435c-aebe-ff6205bb63ab admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:04.844853 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abcdb3-2154-435c-aebe-ff6205bb63ab admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:04.935289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c839740-76fa-44f2-8e0f-9538bd1968bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:04.936218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c839740-76fa-44f2-8e0f-9538bd1968bd admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:04.938347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c839740-76fa-44f2-8e0f-9538bd1968bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:04.938347 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c839740-76fa-44f2-8e0f-9538bd1968bd admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:04.938754 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c839740-76fa-44f2-8e0f-9538bd1968bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:04.941617 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c839740-76fa-44f2-8e0f-9538bd1968bd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:04.950896 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c860f3b-49c3-430f-b837-b6c4c184fc9f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:04.958998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0c860f3b-49c3-430f-b837-b6c4c184fc9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:04.973709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1402/2736] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:01:04 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:04.974604 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c839740-76fa-44f2-8e0f-9538bd1968bd admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:04.985417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c839740-76fa-44f2-8e0f-9538bd1968bd admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:04.987957 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c860f3b-49c3-430f-b837-b6c4c184fc9f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:04.990299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd6659-3378-44fd-88f7-feea90080bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:04.990960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd6659-3378-44fd-88f7-feea90080bff admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:04.991240 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd6659-3378-44fd-88f7-feea90080bff admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:04.991512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd6659-3378-44fd-88f7-feea90080bff admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:04.992238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd6659-3378-44fd-88f7-feea90080bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:04.992850 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:04.993811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:04.993937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd6659-3378-44fd-88f7-feea90080bff admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:04.994228 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:04.994573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] PATH_INFO: `/v3/projects/a292f6648e3c4f6abef5aa1cf5fbb389` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:04.995875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a292f6648e3c4f6abef5aa1cf5fbb389)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:04.998290 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:05.005474 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1403/2737] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:05.009259 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd6659-3378-44fd-88f7-feea90080bff admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:05.014977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:05.021133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd6659-3378-44fd-88f7-feea90080bff admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:05.027835 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75503258-7dfc-49c8-aa20-fd4f3d32cf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:05.028537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75503258-7dfc-49c8-aa20-fd4f3d32cf8c admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.028819 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75503258-7dfc-49c8-aa20-fd4f3d32cf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.029193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75503258-7dfc-49c8-aa20-fd4f3d32cf8c admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.029962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75503258-7dfc-49c8-aa20-fd4f3d32cf8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:05.031551 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75503258-7dfc-49c8-aa20-fd4f3d32cf8c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:05.046242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1404/2738] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:01:05.047692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75503258-7dfc-49c8-aa20-fd4f3d32cf8c admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:05.052529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a292f6648e3c4f6abef5aa1cf5fbb389', 'request_id': 'req-a5168f61-b30e-40e7-9741-f240522fcea3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:05.057730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75503258-7dfc-49c8-aa20-fd4f3d32cf8c admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:05.061410 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a292f6648e3c4f6abef5aa1cf5fbb389', 'request_id': 'req-a5168f61-b30e-40e7-9741-f240522fcea3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:05.067534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a292f6648e3c4f6abef5aa1cf5fbb389', 'request_id': 'req-a5168f61-b30e-40e7-9741-f240522fcea3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:05.081173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] The token cache is being invalidate because project a292f6648e3c4f6abef5aa1cf5fbb389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:01:05.081511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:05.082282 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a5168f61-b30e-40e7-9741-f240522fcea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:05.083754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1335/2739] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:01:04 2022] DELETE /identity/v3/projects/a292f6648e3c4f6abef5aa1cf5fbb389 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:01:05.088345 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1a07b-566c-4b2e-94cf-95fffa8cd1a2 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.088635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1a07b-566c-4b2e-94cf-95fffa8cd1a2 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.088910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1a07b-566c-4b2e-94cf-95fffa8cd1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.108286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-34f1a07b-566c-4b2e-94cf-95fffa8cd1a2 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:05.109558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-34f1a07b-566c-4b2e-94cf-95fffa8cd1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:05.120045 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafe6ea-00de-45ce-b38c-c85dae703ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:05.120956 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafe6ea-00de-45ce-b38c-c85dae703ffa admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.121226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafe6ea-00de-45ce-b38c-c85dae703ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.121609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafe6ea-00de-45ce-b38c-c85dae703ffa admin admin] PATH_INFO: `/v3/projects/60fdf04148f548f1b9fe86ed038e839f/users/cf1d05ee73d4414eb51e2a7be838e28f/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.139311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafe6ea-00de-45ce-b38c-c85dae703ffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=60fdf04148f548f1b9fe86ed038e839f, user_id=cf1d05ee73d4414eb51e2a7be838e28f, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:05.141640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafe6ea-00de-45ce-b38c-c85dae703ffa admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:05.141817 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:05.156878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1405/2740] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:01:05 2022] PUT /identity/v3/projects/60fdf04148f548f1b9fe86ed038e839f/users/cf1d05ee73d4414eb51e2a7be838e28f/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:01:05.161954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db2f3b-3256-40e2-b64d-79a21bc5e8e7 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.162250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db2f3b-3256-40e2-b64d-79a21bc5e8e7 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.162538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db2f3b-3256-40e2-b64d-79a21bc5e8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.176880 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1336/2741] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:01:05.181536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-30db2f3b-3256-40e2-b64d-79a21bc5e8e7 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:05.182906 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-30db2f3b-3256-40e2-b64d-79a21bc5e8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:05.241938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1181-e40e-401d-88cd-21054cde3a11 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.242546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1181-e40e-401d-88cd-21054cde3a11 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.242546 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1181-e40e-401d-88cd-21054cde3a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.255595 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1406/2742] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:01:05.261675 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-772a1181-e40e-401d-88cd-21054cde3a11 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:05.262980 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-772a1181-e40e-401d-88cd-21054cde3a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:05.265016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db2f3b-3256-40e2-b64d-79a21bc5e8e7 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:05.266459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-30db2f3b-3256-40e2-b64d-79a21bc5e8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:05.271614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1337/2743] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:01:05.276393 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a1181-e40e-401d-88cd-21054cde3a11 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:05.285501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a1181-e40e-401d-88cd-21054cde3a11 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:05.290243 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c910a-362f-43aa-ba58-75c89ebb942d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:05.290892 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c910a-362f-43aa-ba58-75c89ebb942d admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.291168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c910a-362f-43aa-ba58-75c89ebb942d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.291435 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c910a-362f-43aa-ba58-75c89ebb942d admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.292248 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c910a-362f-43aa-ba58-75c89ebb942d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:01:05.292555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c910a-362f-43aa-ba58-75c89ebb942d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:05.294809 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c910a-362f-43aa-ba58-75c89ebb942d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:05.307085 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1338/2744] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 21:01:05.312024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c910a-362f-43aa-ba58-75c89ebb942d admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:05.321561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c910a-362f-43aa-ba58-75c89ebb942d admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:05.327572 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fe816-a908-4185-9265-91f6be1611e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:05.328258 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fe816-a908-4185-9265-91f6be1611e5 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.328740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fe816-a908-4185-9265-91f6be1611e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.329015 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fe816-a908-4185-9265-91f6be1611e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.329965 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fe816-a908-4185-9265-91f6be1611e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:05.330182 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db2f3b-3256-40e2-b64d-79a21bc5e8e7 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:05.333386 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fe816-a908-4185-9265-91f6be1611e5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:05.334144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476675f-9555-49ad-b337-3870a03299ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:05.334800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476675f-9555-49ad-b337-3870a03299ed service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.335087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476675f-9555-49ad-b337-3870a03299ed service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.335430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476675f-9555-49ad-b337-3870a03299ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.345522 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476675f-9555-49ad-b337-3870a03299ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:05.348414 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476675f-9555-49ad-b337-3870a03299ed service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:05.360109 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1407/2745] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:01:05.370916 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1339/2746] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:01:05 2022] POST /identity/v3/projects => generated 361 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:01:05.376196 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476675f-9555-49ad-b337-3870a03299ed service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:05.388078 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476675f-9555-49ad-b337-3870a03299ed service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:05.402764 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:05.451387 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a328db0-63c4-498c-ac2a-cf70f5e0868e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:05.452079 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a328db0-63c4-498c-ac2a-cf70f5e0868e admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.452349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a328db0-63c4-498c-ac2a-cf70f5e0868e admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.452623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a328db0-63c4-498c-ac2a-cf70f5e0868e admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.453318 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a328db0-63c4-498c-ac2a-cf70f5e0868e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:05.455297 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a328db0-63c4-498c-ac2a-cf70f5e0868e admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:05.476490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1408/2747] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:01:05 2022] POST /identity/v3/users => generated 281 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:05.480490 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fe816-a908-4185-9265-91f6be1611e5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:05.492914 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fe816-a908-4185-9265-91f6be1611e5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:05.498162 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc58b2-adf2-4fae-b1f5-a09455398887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:05.499230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc58b2-adf2-4fae-b1f5-a09455398887 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.499844 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc58b2-adf2-4fae-b1f5-a09455398887 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.500431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc58b2-adf2-4fae-b1f5-a09455398887 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.502736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc58b2-adf2-4fae-b1f5-a09455398887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:05.504199 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc58b2-adf2-4fae-b1f5-a09455398887 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:05.519476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1340/2748] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:05.520573 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a328db0-63c4-498c-ac2a-cf70f5e0868e admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:05.531865 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a328db0-63c4-498c-ac2a-cf70f5e0868e admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:05.537583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e5015-33e4-4739-82ee-04d5d1e0f647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:05.538391 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e5015-33e4-4739-82ee-04d5d1e0f647 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.538800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e5015-33e4-4739-82ee-04d5d1e0f647 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.539143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e5015-33e4-4739-82ee-04d5d1e0f647 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.539943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e5015-33e4-4739-82ee-04d5d1e0f647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:05.541852 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e5015-33e4-4739-82ee-04d5d1e0f647 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:05.555388 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1409/2749] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:05.559619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc58b2-adf2-4fae-b1f5-a09455398887 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:05.570341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc58b2-adf2-4fae-b1f5-a09455398887 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:05.585644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630a540-a1ac-49b8-b184-487d3e44f4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:05.586805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630a540-a1ac-49b8-b184-487d3e44f4ae admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.587106 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630a540-a1ac-49b8-b184-487d3e44f4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.587389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630a540-a1ac-49b8-b184-487d3e44f4ae admin admin] PATH_INFO: `/v3/projects/270f584e1a884fbd92e751872ab1d7e7/users/db4bb1df25b040869a2a7bf0e48a312e/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.641580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630a540-a1ac-49b8-b184-487d3e44f4ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=270f584e1a884fbd92e751872ab1d7e7, user_id=db4bb1df25b040869a2a7bf0e48a312e, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:05.646505 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630a540-a1ac-49b8-b184-487d3e44f4ae admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:05.671091 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1341/2750] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:01:05 2022] PUT /identity/v3/projects/270f584e1a884fbd92e751872ab1d7e7/users/db4bb1df25b040869a2a7bf0e48a312e/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:01:05.681562 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f8465-b469-46f7-bd09-c5cf0f279e79 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.681907 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f8465-b469-46f7-bd09-c5cf0f279e79 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.682227 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f8465-b469-46f7-bd09-c5cf0f279e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.717191 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be84df-d1c2-4cdf-9f56-e5c9f8b61cb8 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.717610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be84df-d1c2-4cdf-9f56-e5c9f8b61cb8 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.717996 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be84df-d1c2-4cdf-9f56-e5c9f8b61cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.721039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-194f8465-b469-46f7-bd09-c5cf0f279e79 None None] MFA Rules not processed for user `db4bb1df25b040869a2a7bf0e48a312e`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:05.723050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-194f8465-b469-46f7-bd09-c5cf0f279e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:05.780323 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-d4be84df-d1c2-4cdf-9f56-e5c9f8b61cb8 None None] MFA Rules not processed for user `2711df504d424fa0b05cd16d7fd5acfb`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:05.781624 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d4be84df-d1c2-4cdf-9f56-e5c9f8b61cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:05.820417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1410/2751] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:05.858921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f8465-b469-46f7-bd09-c5cf0f279e79 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:05.860368 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-194f8465-b469-46f7-bd09-c5cf0f279e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:05.899271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1342/2752] 10.222.0.70 () {58 vars in 967 bytes} [Tue Jul 19 21:01:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:05.944695 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f8465-b469-46f7-bd09-c5cf0f279e79 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:05.949046 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c548067-59f4-42b8-97ff-deddd01dbb85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:05.950817 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548067-59f4-42b8-97ff-deddd01dbb85 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:05.951210 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548067-59f4-42b8-97ff-deddd01dbb85 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:05.951589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548067-59f4-42b8-97ff-deddd01dbb85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:05.962862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be84df-d1c2-4cdf-9f56-e5c9f8b61cb8 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:05.963698 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c548067-59f4-42b8-97ff-deddd01dbb85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:05.964419 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-d4be84df-d1c2-4cdf-9f56-e5c9f8b61cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:05.965429 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c548067-59f4-42b8-97ff-deddd01dbb85 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:05.977584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1411/2753] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:01:05.994069 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c548067-59f4-42b8-97ff-deddd01dbb85 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:06.012756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:06.049840 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:06.103668 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c548067-59f4-42b8-97ff-deddd01dbb85 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:06.109305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be84df-d1c2-4cdf-9f56-e5c9f8b61cb8 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:06.110250 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:06.111188 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:06.111624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:06.111989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] PATH_INFO: `/v3/users/db4bb1df25b040869a2a7bf0e48a312e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:06.113799 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db4bb1df25b040869a2a7bf0e48a312e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:06.115909 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:06.118318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104f19c-70e7-407c-840a-06b017273472 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '334a380fa68240c79a72746ee4a5f9dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:06.119473 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104f19c-70e7-407c-840a-06b017273472 service swift] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:06.120065 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104f19c-70e7-407c-840a-06b017273472 service swift] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:06.120644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104f19c-70e7-407c-840a-06b017273472 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:06.146617 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104f19c-70e7-407c-840a-06b017273472 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:06.151790 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104f19c-70e7-407c-840a-06b017273472 service swift] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:06.171350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1343/2754] 10.222.0.70 () {60 vars in 1384 bytes} [Tue Jul 19 21:01:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 21:01:06.229798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db4bb1df25b040869a2a7bf0e48a312e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:06.271367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db4bb1df25b040869a2a7bf0e48a312e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:06.286092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db4bb1df25b040869a2a7bf0e48a312e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:06.287634 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db4bb1df25b040869a2a7bf0e48a312e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:06.305292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db4bb1df25b040869a2a7bf0e48a312e'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:06.326545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1412/2755] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:01:05 2022] DELETE /identity/v3/users/db4bb1df25b040869a2a7bf0e48a312e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:01:06.328478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cae0f-7caa-49b5-bcef-3b7b062cad69 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:06.329000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cae0f-7caa-49b5-bcef-3b7b062cad69 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:06.329254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cae0f-7caa-49b5-bcef-3b7b062cad69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:06.354697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-f03cae0f-7caa-49b5-bcef-3b7b062cad69 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:06.357230 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-f03cae0f-7caa-49b5-bcef-3b7b062cad69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:06.463391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1344/2756] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:06.476034 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:06.479714 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:06.581244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24777597-1715-463d-8ea8-68a2d58fef40 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:06.590348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c52fdc-9f2e-4940-934d-a59998026d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:06.591883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c52fdc-9f2e-4940-934d-a59998026d47 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:06.592370 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c52fdc-9f2e-4940-934d-a59998026d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:06.593241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c52fdc-9f2e-4940-934d-a59998026d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:06.613978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c52fdc-9f2e-4940-934d-a59998026d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:06.615615 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c52fdc-9f2e-4940-934d-a59998026d47 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:06.631375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1413/2757] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:06.776122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cae0f-7caa-49b5-bcef-3b7b062cad69 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:06.789177 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cae0f-7caa-49b5-bcef-3b7b062cad69 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:06.797643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319b019-deba-49b5-9b7c-05a73b2b4984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:06.798580 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319b019-deba-49b5-9b7c-05a73b2b4984 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:06.799052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319b019-deba-49b5-9b7c-05a73b2b4984 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:06.799498 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319b019-deba-49b5-9b7c-05a73b2b4984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:06.801997 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-b319b019-deba-49b5-9b7c-05a73b2b4984 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:06.822326 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319b019-deba-49b5-9b7c-05a73b2b4984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:06.825513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319b019-deba-49b5-9b7c-05a73b2b4984 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:06.839089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1345/2758] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:07.302183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c52fdc-9f2e-4940-934d-a59998026d47 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:07.303708 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-42c52fdc-9f2e-4940-934d-a59998026d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:07.327261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c52fdc-9f2e-4940-934d-a59998026d47 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:07.341160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:07.342459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:07.342723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:07.343005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] PATH_INFO: `/v3/projects/270f584e1a884fbd92e751872ab1d7e7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:07.344311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=270f584e1a884fbd92e751872ab1d7e7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:07.346680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:07.410073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:07.414088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '270f584e1a884fbd92e751872ab1d7e7', 'request_id': 'req-a6fd5669-df35-41d6-bbb8-f1e002735873'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:07.426285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '270f584e1a884fbd92e751872ab1d7e7', 'request_id': 'req-a6fd5669-df35-41d6-bbb8-f1e002735873'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:07.434375 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '270f584e1a884fbd92e751872ab1d7e7', 'request_id': 'req-a6fd5669-df35-41d6-bbb8-f1e002735873'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:07.450619 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] The token cache is being invalidate because project 270f584e1a884fbd92e751872ab1d7e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:01:07.450919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:07.455018 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a6fd5669-df35-41d6-bbb8-f1e002735873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:07.457112 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1414/2759] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:01:07 2022] DELETE /identity/v3/projects/270f584e1a884fbd92e751872ab1d7e7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:07.462443 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3696e5-13ea-4d88-b1e3-c615949cf959 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:07.462955 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3696e5-13ea-4d88-b1e3-c615949cf959 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:07.463115 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3696e5-13ea-4d88-b1e3-c615949cf959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:07.485740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-4a3696e5-13ea-4d88-b1e3-c615949cf959 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:07.487280 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4a3696e5-13ea-4d88-b1e3-c615949cf959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:07.587783 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1346/2760] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:07.595913 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f28b1-b2d9-41e0-81ed-27d03ba09631 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:07.597257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f28b1-b2d9-41e0-81ed-27d03ba09631 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:07.597826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f28b1-b2d9-41e0-81ed-27d03ba09631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:07.625841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-6f5f28b1-b2d9-41e0-81ed-27d03ba09631 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:07.627227 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6f5f28b1-b2d9-41e0-81ed-27d03ba09631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:07.636767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1415/2761] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:07.643118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3696e5-13ea-4d88-b1e3-c615949cf959 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:07.658164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3696e5-13ea-4d88-b1e3-c615949cf959 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:07.663879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017eaaa6-2df5-40c1-ac6c-b2427518b84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:07.663879 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017eaaa6-2df5-40c1-ac6c-b2427518b84e admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:07.664157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017eaaa6-2df5-40c1-ac6c-b2427518b84e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:07.664434 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017eaaa6-2df5-40c1-ac6c-b2427518b84e admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:07.665458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017eaaa6-2df5-40c1-ac6c-b2427518b84e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:01:07.665588 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017eaaa6-2df5-40c1-ac6c-b2427518b84e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:07.667301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017eaaa6-2df5-40c1-ac6c-b2427518b84e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:07.685260 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1347/2762] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:01:07.686518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f28b1-b2d9-41e0-81ed-27d03ba09631 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:07.698563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f28b1-b2d9-41e0-81ed-27d03ba09631 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:07.707641 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867a76b-02c2-4fe3-a11a-52435eafad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:07.709586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867a76b-02c2-4fe3-a11a-52435eafad81 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:07.711154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867a76b-02c2-4fe3-a11a-52435eafad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:07.711154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867a76b-02c2-4fe3-a11a-52435eafad81 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:07.711154 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867a76b-02c2-4fe3-a11a-52435eafad81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:07.713348 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867a76b-02c2-4fe3-a11a-52435eafad81 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:07.750380 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1416/2763] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:01:07 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:07.754750 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017eaaa6-2df5-40c1-ac6c-b2427518b84e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:07.768278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017eaaa6-2df5-40c1-ac6c-b2427518b84e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:07.786685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=100 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:07.888928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42ac4d-77da-4bbf-9e1a-a7ae1cd91ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:07.890025 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42ac4d-77da-4bbf-9e1a-a7ae1cd91ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:07.890457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42ac4d-77da-4bbf-9e1a-a7ae1cd91ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:07.891291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42ac4d-77da-4bbf-9e1a-a7ae1cd91ba9 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:07.895902 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42ac4d-77da-4bbf-9e1a-a7ae1cd91ba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:07.898463 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42ac4d-77da-4bbf-9e1a-a7ae1cd91ba9 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:07.952266 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1348/2764] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:01:07 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:07.953989 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867a76b-02c2-4fe3-a11a-52435eafad81 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:07.973916 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867a76b-02c2-4fe3-a11a-52435eafad81 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:07.984778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feddae-007e-4ea0-9ed5-3bd1e7864ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:07.985549 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feddae-007e-4ea0-9ed5-3bd1e7864ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:07.985842 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feddae-007e-4ea0-9ed5-3bd1e7864ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:07.986146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feddae-007e-4ea0-9ed5-3bd1e7864ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:07.986875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38feddae-007e-4ea0-9ed5-3bd1e7864ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:07.988624 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38feddae-007e-4ea0-9ed5-3bd1e7864ed4 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:08.011955 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1417/2765] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:08.017143 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42ac4d-77da-4bbf-9e1a-a7ae1cd91ba9 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:08.043333 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42ac4d-77da-4bbf-9e1a-a7ae1cd91ba9 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:08.055172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15844a-c230-49a7-9e97-4aad9acd2f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:08.055869 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15844a-c230-49a7-9e97-4aad9acd2f94 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:08.056160 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15844a-c230-49a7-9e97-4aad9acd2f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:08.056890 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15844a-c230-49a7-9e97-4aad9acd2f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:08.057574 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15844a-c230-49a7-9e97-4aad9acd2f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:08.059669 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15844a-c230-49a7-9e97-4aad9acd2f94 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:08.072096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1349/2766] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:08.077523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feddae-007e-4ea0-9ed5-3bd1e7864ed4 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:08.091567 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feddae-007e-4ea0-9ed5-3bd1e7864ed4 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:08.100607 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fcad5-e3d7-464e-af70-0a2ac1fbbd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:08.101575 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fcad5-e3d7-464e-af70-0a2ac1fbbd63 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:08.101860 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fcad5-e3d7-464e-af70-0a2ac1fbbd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:08.102593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fcad5-e3d7-464e-af70-0a2ac1fbbd63 admin admin] PATH_INFO: `/v3/projects/f92542b5a8bd462db2d8d583c9480dec/users/f79e9e005f4548c2a5a2c07d24c240e3/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:08.124382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fcad5-e3d7-464e-af70-0a2ac1fbbd63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92542b5a8bd462db2d8d583c9480dec, user_id=f79e9e005f4548c2a5a2c07d24c240e3, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:08.127242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fcad5-e3d7-464e-af70-0a2ac1fbbd63 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:08.148679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1418/2767] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:01:08 2022] PUT /identity/v3/projects/f92542b5a8bd462db2d8d583c9480dec/users/f79e9e005f4548c2a5a2c07d24c240e3/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:08.162306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3580f9-019f-4e84-b189-a33a051b54e0 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:08.162306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3580f9-019f-4e84-b189-a33a051b54e0 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:08.162306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3580f9-019f-4e84-b189-a33a051b54e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:08.209581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-fc3580f9-019f-4e84-b189-a33a051b54e0 None None] MFA Rules not processed for user `f79e9e005f4548c2a5a2c07d24c240e3`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:08.211262 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-fc3580f9-019f-4e84-b189-a33a051b54e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:08.339659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1350/2768] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:08.347490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fcad5-e3d7-464e-af70-0a2ac1fbbd63 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:08.438954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fcad5-e3d7-464e-af70-0a2ac1fbbd63 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:08.445050 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d4bb6-0b7f-4832-bcb0-1e6f8e8d2159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:08.445777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d4bb6-0b7f-4832-bcb0-1e6f8e8d2159 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:08.446062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d4bb6-0b7f-4832-bcb0-1e6f8e8d2159 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:08.446339 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d4bb6-0b7f-4832-bcb0-1e6f8e8d2159 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:08.447035 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d4bb6-0b7f-4832-bcb0-1e6f8e8d2159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:08.449071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d4bb6-0b7f-4832-bcb0-1e6f8e8d2159 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:08.504977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1419/2769] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:01:08 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:08.506625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3580f9-019f-4e84-b189-a33a051b54e0 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:08.532502 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3580f9-019f-4e84-b189-a33a051b54e0 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:08.630676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cfc93-c24c-460c-b832-a3f340dc59bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:08.631642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cfc93-c24c-460c-b832-a3f340dc59bd admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:08.632096 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cfc93-c24c-460c-b832-a3f340dc59bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:08.632661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cfc93-c24c-460c-b832-a3f340dc59bd admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:08.633413 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cfc93-c24c-460c-b832-a3f340dc59bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:08.635352 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cfc93-c24c-460c-b832-a3f340dc59bd admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:08.667083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1351/2770] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:01:08 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:08.672990 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d4bb6-0b7f-4832-bcb0-1e6f8e8d2159 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:08.684266 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d4bb6-0b7f-4832-bcb0-1e6f8e8d2159 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:08.695656 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de8d80-7c92-477e-aded-34f060cbde7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:08.696357 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de8d80-7c92-477e-aded-34f060cbde7b admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:08.696801 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de8d80-7c92-477e-aded-34f060cbde7b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:08.697090 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de8d80-7c92-477e-aded-34f060cbde7b admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:08.697872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de8d80-7c92-477e-aded-34f060cbde7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:08.699630 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de8d80-7c92-477e-aded-34f060cbde7b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:08.722707 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1420/2771] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:08.724507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cfc93-c24c-460c-b832-a3f340dc59bd admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:08.739438 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cfc93-c24c-460c-b832-a3f340dc59bd admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:08.745835 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ee413-7f6e-46af-a8ea-f1229dbd6915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:08.746982 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ee413-7f6e-46af-a8ea-f1229dbd6915 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:08.747486 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ee413-7f6e-46af-a8ea-f1229dbd6915 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:08.748010 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ee413-7f6e-46af-a8ea-f1229dbd6915 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:08.749166 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ee413-7f6e-46af-a8ea-f1229dbd6915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:08.751142 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ee413-7f6e-46af-a8ea-f1229dbd6915 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:08.776127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1352/2772] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:08.781802 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de8d80-7c92-477e-aded-34f060cbde7b admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:08.797084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de8d80-7c92-477e-aded-34f060cbde7b admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:08.814920 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a8b04-3906-432b-8807-21033fd544eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:08.816511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a8b04-3906-432b-8807-21033fd544eb admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:08.817608 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a8b04-3906-432b-8807-21033fd544eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:08.818107 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a8b04-3906-432b-8807-21033fd544eb admin admin] PATH_INFO: `/v3/projects/aec2f4ac3a1246c7a65da4645d89392b/users/2978b858c03043eba971cc53c12c198c/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:08.848826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a8b04-3906-432b-8807-21033fd544eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec2f4ac3a1246c7a65da4645d89392b, user_id=2978b858c03043eba971cc53c12c198c, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:08.852300 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a8b04-3906-432b-8807-21033fd544eb admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:08.880225 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1421/2773] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:01:08 2022] PUT /identity/v3/projects/aec2f4ac3a1246c7a65da4645d89392b/users/2978b858c03043eba971cc53c12c198c/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:08.891383 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f901b7b-88e5-45a8-be00-a2ec470ebaff None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:08.891763 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f901b7b-88e5-45a8-be00-a2ec470ebaff None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:08.891940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f901b7b-88e5-45a8-be00-a2ec470ebaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:08.900238 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:08.940012 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-5f901b7b-88e5-45a8-be00-a2ec470ebaff None None] MFA Rules not processed for user `2978b858c03043eba971cc53c12c198c`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:08.941339 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5f901b7b-88e5-45a8-be00-a2ec470ebaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:09.071286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a8b04-3906-432b-8807-21033fd544eb admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:09.071532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1353/2774] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:09.102085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:09.181633 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a8b04-3906-432b-8807-21033fd544eb admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:09.190199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d90edf-7efd-4e6f-bfa1-f2296157ab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:09.191122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d90edf-7efd-4e6f-bfa1-f2296157ab89 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:09.192772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d90edf-7efd-4e6f-bfa1-f2296157ab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:09.193198 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d90edf-7efd-4e6f-bfa1-f2296157ab89 admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:09.196366 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d90edf-7efd-4e6f-bfa1-f2296157ab89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:09.198778 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d90edf-7efd-4e6f-bfa1-f2296157ab89 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:09.246682 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1422/2775] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:01:09 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:09.251652 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f901b7b-88e5-45a8-be00-a2ec470ebaff None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:09.271847 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f901b7b-88e5-45a8-be00-a2ec470ebaff None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:09.435770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143b6a3-d6ae-4cdc-866d-b3cc84b9ab46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:09.436645 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143b6a3-d6ae-4cdc-866d-b3cc84b9ab46 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:09.436938 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143b6a3-d6ae-4cdc-866d-b3cc84b9ab46 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:09.437219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143b6a3-d6ae-4cdc-866d-b3cc84b9ab46 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:09.439676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143b6a3-d6ae-4cdc-866d-b3cc84b9ab46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:09.441592 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143b6a3-d6ae-4cdc-866d-b3cc84b9ab46 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:09.487017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1354/2776] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:01:09 2022] POST /identity/v3/users => generated 299 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:09.491029 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d90edf-7efd-4e6f-bfa1-f2296157ab89 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:09.506984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d90edf-7efd-4e6f-bfa1-f2296157ab89 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:09.522804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195a0db-77ef-47f6-b6dc-2eb3081375f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:09.523450 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195a0db-77ef-47f6-b6dc-2eb3081375f5 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:09.523740 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195a0db-77ef-47f6-b6dc-2eb3081375f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:09.524026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195a0db-77ef-47f6-b6dc-2eb3081375f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:09.524767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195a0db-77ef-47f6-b6dc-2eb3081375f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:09.530893 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195a0db-77ef-47f6-b6dc-2eb3081375f5 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:09.562157 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1423/2777] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:09.563523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143b6a3-d6ae-4cdc-866d-b3cc84b9ab46 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:09.577610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143b6a3-d6ae-4cdc-866d-b3cc84b9ab46 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:09.589976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154410e-823f-4b36-8654-22dd2f337347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:09.590934 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154410e-823f-4b36-8654-22dd2f337347 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:09.591217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154410e-823f-4b36-8654-22dd2f337347 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:09.591496 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154410e-823f-4b36-8654-22dd2f337347 admin admin] PATH_INFO: `/v3/projects/a0da68c294984a7daa71c8b64cb7cdb6/users/8a5e7674e94e41ea9aecef240ad18954/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:09.642363 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154410e-823f-4b36-8654-22dd2f337347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0da68c294984a7daa71c8b64cb7cdb6, user_id=8a5e7674e94e41ea9aecef240ad18954, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:09.645667 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154410e-823f-4b36-8654-22dd2f337347 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:09.684122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1355/2778] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:01:09 2022] PUT /identity/v3/projects/a0da68c294984a7daa71c8b64cb7cdb6/users/8a5e7674e94e41ea9aecef240ad18954/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:09.688539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195a0db-77ef-47f6-b6dc-2eb3081375f5 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:09.702488 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195a0db-77ef-47f6-b6dc-2eb3081375f5 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:09.843550 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ee1ef-e139-44c7-bf49-d60bf8455558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:09.844789 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee1ef-e139-44c7-bf49-d60bf8455558 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:09.845846 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee1ef-e139-44c7-bf49-d60bf8455558 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:09.846628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ee1ef-e139-44c7-bf49-d60bf8455558 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:09.847626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ee1ef-e139-44c7-bf49-d60bf8455558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:09.850104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ee1ef-e139-44c7-bf49-d60bf8455558 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:09.884718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1424/2779] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:09.889650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154410e-823f-4b36-8654-22dd2f337347 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:09.920324 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154410e-823f-4b36-8654-22dd2f337347 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:09.928346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d8154-c6b9-49b7-94f8-e4b77ae99bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:09.929634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d8154-c6b9-49b7-94f8-e4b77ae99bef admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:09.930130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d8154-c6b9-49b7-94f8-e4b77ae99bef admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:09.930636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d8154-c6b9-49b7-94f8-e4b77ae99bef admin admin] PATH_INFO: `/v3/projects/a0da68c294984a7daa71c8b64cb7cdb6/users/8a5e7674e94e41ea9aecef240ad18954/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:09.933682 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d8154-c6b9-49b7-94f8-e4b77ae99bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0da68c294984a7daa71c8b64cb7cdb6, user_id=8a5e7674e94e41ea9aecef240ad18954, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:09.937610 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d8154-c6b9-49b7-94f8-e4b77ae99bef admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:09.971683 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1356/2780] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:01:09 2022] PUT /identity/v3/projects/a0da68c294984a7daa71c8b64cb7cdb6/users/8a5e7674e94e41ea9aecef240ad18954/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:01:09.976834 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605f821-611d-484d-b721-9419fbbae8cf None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:09.977295 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605f821-611d-484d-b721-9419fbbae8cf None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:09.977891 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605f821-611d-484d-b721-9419fbbae8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:10.056437 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-9605f821-611d-484d-b721-9419fbbae8cf None None] MFA Rules not processed for user `8a5e7674e94e41ea9aecef240ad18954`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:10.058066 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9605f821-611d-484d-b721-9419fbbae8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:10.225047 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1425/2781] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:10.235071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d8154-c6b9-49b7-94f8-e4b77ae99bef admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:10.237285 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-500d8154-c6b9-49b7-94f8-e4b77ae99bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:10.308164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:10.323019 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d8154-c6b9-49b7-94f8-e4b77ae99bef admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:10.329654 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56ff7a-2aa6-4d2c-86b8-c0f083ad0a4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:10.330346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56ff7a-2aa6-4d2c-86b8-c0f083ad0a4e service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:10.330635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56ff7a-2aa6-4d2c-86b8-c0f083ad0a4e service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:10.330924 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56ff7a-2aa6-4d2c-86b8-c0f083ad0a4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:10.464655 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56ff7a-2aa6-4d2c-86b8-c0f083ad0a4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:10.466957 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56ff7a-2aa6-4d2c-86b8-c0f083ad0a4e service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:10.484992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1357/2782] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:11.542547 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2977c6-08ef-461f-bab6-b0f1a8a331b1 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:11.543020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2977c6-08ef-461f-bab6-b0f1a8a331b1 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:11.543257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2977c6-08ef-461f-bab6-b0f1a8a331b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:11.667886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-1d2977c6-08ef-461f-bab6-b0f1a8a331b1 None None] MFA Rules not processed for user `cf1d05ee73d4414eb51e2a7be838e28f`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:11.670270 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1d2977c6-08ef-461f-bab6-b0f1a8a331b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:11.684618 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:11.866289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1426/2783] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:11.875331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56ff7a-2aa6-4d2c-86b8-c0f083ad0a4e service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:11.877723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0b56ff7a-2aa6-4d2c-86b8-c0f083ad0a4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:12.086271 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56ff7a-2aa6-4d2c-86b8-c0f083ad0a4e service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:12.094074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b01724-7a66-443a-8326-f9061656315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:12.094868 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b01724-7a66-443a-8326-f9061656315f admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:12.095216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b01724-7a66-443a-8326-f9061656315f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:12.095563 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b01724-7a66-443a-8326-f9061656315f admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:12.096697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b01724-7a66-443a-8326-f9061656315f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:12.106588 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b01724-7a66-443a-8326-f9061656315f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:12.145412 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2977c6-08ef-461f-bab6-b0f1a8a331b1 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:12.150268 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1d2977c6-08ef-461f-bab6-b0f1a8a331b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:12.165748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1358/2784] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:01:11 2022] POST /identity/v3/projects => generated 377 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:12.171736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b01724-7a66-443a-8326-f9061656315f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:12.293678 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b01724-7a66-443a-8326-f9061656315f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:12.310127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddd324-67ce-493f-973f-ac9de13dca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:12.316280 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddd324-67ce-493f-973f-ac9de13dca5d admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:12.317470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddd324-67ce-493f-973f-ac9de13dca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:12.327542 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddd324-67ce-493f-973f-ac9de13dca5d admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:12.328676 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddd324-67ce-493f-973f-ac9de13dca5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:12.332873 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddd324-67ce-493f-973f-ac9de13dca5d admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:12.337008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2977c6-08ef-461f-bab6-b0f1a8a331b1 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:12.420732 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1359/2785] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:01:12 2022] POST /identity/v3/users => generated 295 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:12.421769 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddd324-67ce-493f-973f-ac9de13dca5d admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:12.440660 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddd324-67ce-493f-973f-ac9de13dca5d admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:12.448745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35579c-1774-47da-bf97-f062e2fb1cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:12.450288 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35579c-1774-47da-bf97-f062e2fb1cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:12.450886 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35579c-1774-47da-bf97-f062e2fb1cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:12.451648 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35579c-1774-47da-bf97-f062e2fb1cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:12.453097 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35579c-1774-47da-bf97-f062e2fb1cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:12.473187 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab5b89-0401-4ff5-9921-c087cfeb23cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:12.473993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab5b89-0401-4ff5-9921-c087cfeb23cc service placement] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:12.474373 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab5b89-0401-4ff5-9921-c087cfeb23cc service placement] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:12.474739 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab5b89-0401-4ff5-9921-c087cfeb23cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:12.475931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35579c-1774-47da-bf97-f062e2fb1cc9 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:12.476896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b2ab5b89-0401-4ff5-9921-c087cfeb23cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:12.496473 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab5b89-0401-4ff5-9921-c087cfeb23cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:12.499680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab5b89-0401-4ff5-9921-c087cfeb23cc service placement] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:12.512728 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1427/2786] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:01:12.514132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab5b89-0401-4ff5-9921-c087cfeb23cc service placement] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:12.516884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1360/2787] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:12.528151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab5b89-0401-4ff5-9921-c087cfeb23cc service placement] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:12.534319 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475cd99-2622-40a7-ad53-7769c6bf1125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:12.535359 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475cd99-2622-40a7-ad53-7769c6bf1125 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:12.535750 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475cd99-2622-40a7-ad53-7769c6bf1125 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:12.536133 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475cd99-2622-40a7-ad53-7769c6bf1125 admin admin] PATH_INFO: `/v3/projects/0b9d6363aed94f1e8407fa51bf167f08/users/1908f29da22e47fc9071d1cfb495c561/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:12.537190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35579c-1774-47da-bf97-f062e2fb1cc9 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:12.542146 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:12.571523 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475cd99-2622-40a7-ad53-7769c6bf1125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9d6363aed94f1e8407fa51bf167f08, user_id=1908f29da22e47fc9071d1cfb495c561, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:12.575640 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475cd99-2622-40a7-ad53-7769c6bf1125 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:12.603555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1428/2788] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:01:12 2022] PUT /identity/v3/projects/0b9d6363aed94f1e8407fa51bf167f08/users/1908f29da22e47fc9071d1cfb495c561/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:12.610943 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475cd99-2622-40a7-ad53-7769c6bf1125 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:12.634358 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35579c-1774-47da-bf97-f062e2fb1cc9 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:12.641464 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:12.738433 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475cd99-2622-40a7-ad53-7769c6bf1125 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:12.744729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0b680-a0b2-49b5-b41d-fcb08964d2d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:12.745062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69bc6fb-2222-46a2-a622-f7ebb1794e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:12.745615 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0b680-a0b2-49b5-b41d-fcb08964d2d7 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:12.745810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bc6fb-2222-46a2-a622-f7ebb1794e15 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:12.745923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0b680-a0b2-49b5-b41d-fcb08964d2d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:12.746097 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bc6fb-2222-46a2-a622-f7ebb1794e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:12.746246 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0b680-a0b2-49b5-b41d-fcb08964d2d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:12.746392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bc6fb-2222-46a2-a622-f7ebb1794e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:12.749021 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69bc6fb-2222-46a2-a622-f7ebb1794e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:12.750767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69bc6fb-2222-46a2-a622-f7ebb1794e15 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:12.764756 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1429/2789] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:12.769897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69bc6fb-2222-46a2-a622-f7ebb1794e15 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:12.781929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69bc6fb-2222-46a2-a622-f7ebb1794e15 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:12.788200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867862f-1228-45e7-b3e4-d020fac875c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:12.789994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867862f-1228-45e7-b3e4-d020fac875c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:12.790430 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867862f-1228-45e7-b3e4-d020fac875c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:12.791098 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867862f-1228-45e7-b3e4-d020fac875c1 admin admin] PATH_INFO: `/v3/projects/0b9d6363aed94f1e8407fa51bf167f08/users/1908f29da22e47fc9071d1cfb495c561/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:12.794249 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867862f-1228-45e7-b3e4-d020fac875c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9d6363aed94f1e8407fa51bf167f08, user_id=1908f29da22e47fc9071d1cfb495c561, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:12.797158 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867862f-1228-45e7-b3e4-d020fac875c1 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:12.825470 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1430/2790] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:01:12 2022] PUT /identity/v3/projects/0b9d6363aed94f1e8407fa51bf167f08/users/1908f29da22e47fc9071d1cfb495c561/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:12.839494 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0b680-a0b2-49b5-b41d-fcb08964d2d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:12.841784 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0b680-a0b2-49b5-b41d-fcb08964d2d7 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:12.867307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1361/2791] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:13.841511 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867862f-1228-45e7-b3e4-d020fac875c1 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:13.972244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867862f-1228-45e7-b3e4-d020fac875c1 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:13.978629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30348936-9b47-411e-8e73-38aa85396adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:13.979686 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348936-9b47-411e-8e73-38aa85396adb service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:13.980145 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348936-9b47-411e-8e73-38aa85396adb service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:13.980571 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348936-9b47-411e-8e73-38aa85396adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:14.101746 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30348936-9b47-411e-8e73-38aa85396adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:14.103863 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30348936-9b47-411e-8e73-38aa85396adb service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:14.122949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1431/2792] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:01:17.593900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0b680-a0b2-49b5-b41d-fcb08964d2d7 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:17.595867 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-eff0b680-a0b2-49b5-b41d-fcb08964d2d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:17.625216 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0b680-a0b2-49b5-b41d-fcb08964d2d7 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:17.721432 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:17.722771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:17.723124 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:17.723481 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] PATH_INFO: `/v3/users/c0bbce46598c465bb29cb0353f10e3eb` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:17.725071 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0bbce46598c465bb29cb0353f10e3eb)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:17.727534 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:17.748531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:17.814583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0bbce46598c465bb29cb0353f10e3eb'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:17.823318 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0bbce46598c465bb29cb0353f10e3eb'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:17.832717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0bbce46598c465bb29cb0353f10e3eb'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:17.863479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0bbce46598c465bb29cb0353f10e3eb'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:17.866305 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0bbce46598c465bb29cb0353f10e3eb'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:17.876650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1362/2793] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:01:17 2022] DELETE /identity/v3/users/c0bbce46598c465bb29cb0353f10e3eb => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:18.100030 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30348936-9b47-411e-8e73-38aa85396adb service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:18.120371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30348936-9b47-411e-8e73-38aa85396adb service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:18.188071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:18.205179 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:18.206099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:18.206435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:18.206767 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] PATH_INFO: `/v3/projects/3dcf50fca2134c84812de75fed624869` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:18.208014 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dcf50fca2134c84812de75fed624869)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:18.209977 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:18.267008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dcf50fca2134c84812de75fed624869', 'request_id': 'req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:18.278508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dcf50fca2134c84812de75fed624869', 'request_id': 'req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:18.286220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dcf50fca2134c84812de75fed624869', 'request_id': 'req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:18.302125 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] The token cache is being invalidate because project 3dcf50fca2134c84812de75fed624869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:01:18.303185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:18.303929 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:18.310977 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:18.311558 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1432/2794] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:01:18 2022] DELETE /identity/v3/projects/3dcf50fca2134c84812de75fed624869 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:01:18.312843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:18.329586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cab81-c175-48ea-bd4d-2c5c91d77994 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:18.431704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc3e44-8be2-414b-9752-d5104cc8d21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:18.433184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc3e44-8be2-414b-9752-d5104cc8d21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:18.434052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc3e44-8be2-414b-9752-d5104cc8d21e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:18.434306 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc3e44-8be2-414b-9752-d5104cc8d21e admin admin] PATH_INFO: `/v3/users/80b9b704671047439993d6d91170b1b2` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:18.436247 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc3e44-8be2-414b-9752-d5104cc8d21e admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b9b704671047439993d6d91170b1b2)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:18.439437 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc3e44-8be2-414b-9752-d5104cc8d21e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:18.528609 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7ddc3e44-8be2-414b-9752-d5104cc8d21e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b9b704671047439993d6d91170b1b2'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:18.575639 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7ddc3e44-8be2-414b-9752-d5104cc8d21e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b9b704671047439993d6d91170b1b2'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:18.604112 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7ddc3e44-8be2-414b-9752-d5104cc8d21e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b9b704671047439993d6d91170b1b2'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:18.639114 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7ddc3e44-8be2-414b-9752-d5104cc8d21e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b9b704671047439993d6d91170b1b2'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:18.641450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7ddc3e44-8be2-414b-9752-d5104cc8d21e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b9b704671047439993d6d91170b1b2'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:18.659727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1363/2795] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:01:18 2022] DELETE /identity/v3/users/80b9b704671047439993d6d91170b1b2 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:18.953153 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:18.954883 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:18.974589 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2285d2c-2cc3-4a9f-80dc-cc0081374a8d admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:19.070867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:19.072235 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:19.072552 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:19.073910 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] PATH_INFO: `/v3/projects/8631943115184923a07591723a58833e` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:19.076274 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=8631943115184923a07591723a58833e)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:19.080282 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:19.158802 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6178024-58e4-4e74-92de-df1d91848158 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:19.158919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6178024-58e4-4e74-92de-df1d91848158 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:19.159182 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6178024-58e4-4e74-92de-df1d91848158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:19.179436 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8631943115184923a07591723a58833e', 'request_id': 'req-6bce77d4-6978-4b04-9ad3-dd61778217ae'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:19.187244 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:19.193472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8631943115184923a07591723a58833e', 'request_id': 'req-6bce77d4-6978-4b04-9ad3-dd61778217ae'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:19.196695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a6178024-58e4-4e74-92de-df1d91848158 None None] MFA Rules not processed for user `1908f29da22e47fc9071d1cfb495c561`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:19.199078 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a6178024-58e4-4e74-92de-df1d91848158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:19.202445 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8631943115184923a07591723a58833e', 'request_id': 'req-6bce77d4-6978-4b04-9ad3-dd61778217ae'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:19.220365 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] The token cache is being invalidate because project 8631943115184923a07591723a58833e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:01:19.220810 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:19.222483 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:19.224482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1433/2796] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:01:18 2022] DELETE /identity/v3/projects/8631943115184923a07591723a58833e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:19.295944 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1364/2797] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:19.314434 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:19.316701 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:19.341995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:19.473581 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce77d4-6978-4b04-9ad3-dd61778217ae admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:19.492757 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f17d-32a6-4e84-92c5-ead4a8926e53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:19.493899 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f17d-32a6-4e84-92c5-ead4a8926e53 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:19.494548 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f17d-32a6-4e84-92c5-ead4a8926e53 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:19.495424 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f17d-32a6-4e84-92c5-ead4a8926e53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:19.499126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c278f17d-32a6-4e84-92c5-ead4a8926e53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:19.594878 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278f17d-32a6-4e84-92c5-ead4a8926e53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:19.597011 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278f17d-32a6-4e84-92c5-ead4a8926e53 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:19.608930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1434/2798] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:22.795954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6178024-58e4-4e74-92de-df1d91848158 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:22.797540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a6178024-58e4-4e74-92de-df1d91848158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:22.863513 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6178024-58e4-4e74-92de-df1d91848158 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:22.868686 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:22.869612 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:22.872140 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:22.872140 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] PATH_INFO: `/v3/users/f79e9e005f4548c2a5a2c07d24c240e3` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:22.872140 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=f79e9e005f4548c2a5a2c07d24c240e3)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:22.874000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:22.928378 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f79e9e005f4548c2a5a2c07d24c240e3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:22.938322 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f79e9e005f4548c2a5a2c07d24c240e3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:22.947356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f79e9e005f4548c2a5a2c07d24c240e3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:22.963737 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f79e9e005f4548c2a5a2c07d24c240e3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:22.964500 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f79e9e005f4548c2a5a2c07d24c240e3'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:22.971794 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1365/2799] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:01:22 2022] DELETE /identity/v3/users/f79e9e005f4548c2a5a2c07d24c240e3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:22.976289 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f71dc-dc85-4c25-b51e-d359045a1c58 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:22.976780 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f71dc-dc85-4c25-b51e-d359045a1c58 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:22.976909 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f71dc-dc85-4c25-b51e-d359045a1c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:22.995102 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-b35f71dc-dc85-4c25-b51e-d359045a1c58 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:22.997126 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b35f71dc-dc85-4c25-b51e-d359045a1c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:23.073462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1435/2800] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:23.082992 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:23.084723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:23.103928 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:23.145684 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714df84-045c-4b3c-8bec-1221e01179ae admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:23.149342 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280316af-f4b9-4fd3-93d8-f8422fcb326e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:23.150018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280316af-f4b9-4fd3-93d8-f8422fcb326e service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:23.150300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280316af-f4b9-4fd3-93d8-f8422fcb326e service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:23.150576 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280316af-f4b9-4fd3-93d8-f8422fcb326e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:23.160736 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280316af-f4b9-4fd3-93d8-f8422fcb326e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:23.162184 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280316af-f4b9-4fd3-93d8-f8422fcb326e service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:23.173107 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1366/2801] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:23.576992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f71dc-dc85-4c25-b51e-d359045a1c58 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:23.581038 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b35f71dc-dc85-4c25-b51e-d359045a1c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:23.607039 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f71dc-dc85-4c25-b51e-d359045a1c58 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:23.625605 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:23.629876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:23.630171 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:23.630459 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] PATH_INFO: `/v3/projects/f92542b5a8bd462db2d8d583c9480dec` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:23.635048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92542b5a8bd462db2d8d583c9480dec)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:23.641953 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:23.731315 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:23.754428 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92542b5a8bd462db2d8d583c9480dec', 'request_id': 'req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:23.766496 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92542b5a8bd462db2d8d583c9480dec', 'request_id': 'req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:23.775897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92542b5a8bd462db2d8d583c9480dec', 'request_id': 'req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:23.809674 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] The token cache is being invalidate because project f92542b5a8bd462db2d8d583c9480dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:01:23.810303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:23.811277 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:23.816508 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1436/2802] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:01:23 2022] DELETE /identity/v3/projects/f92542b5a8bd462db2d8d583c9480dec => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:23.818229 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280316af-f4b9-4fd3-93d8-f8422fcb326e service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:23.854272 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280316af-f4b9-4fd3-93d8-f8422fcb326e service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:23.984115 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:23.986457 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:23.986805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:23.987095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] PATH_INFO: `/v3/users/2978b858c03043eba971cc53c12c198c` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:23.988789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2978b858c03043eba971cc53c12c198c)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:23.990948 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:24.086171 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2978b858c03043eba971cc53c12c198c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:24.098427 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2978b858c03043eba971cc53c12c198c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:24.108389 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2978b858c03043eba971cc53c12c198c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:24.129193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2978b858c03043eba971cc53c12c198c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:24.130001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2978b858c03043eba971cc53c12c198c'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:24.136870 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1367/2803] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:01:23 2022] DELETE /identity/v3/users/2978b858c03043eba971cc53c12c198c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:24.499709 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:24.513476 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:24.574085 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d8d0f-7f6e-4c41-87b1-ceecca28f4e7 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:24.696901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:24.697950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:24.698346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:24.699257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] PATH_INFO: `/v3/projects/aec2f4ac3a1246c7a65da4645d89392b` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:24.703856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] RBAC: Authorizing `identity:delete_project(project_id=aec2f4ac3a1246c7a65da4645d89392b)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:24.706092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:24.817584 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aec2f4ac3a1246c7a65da4645d89392b', 'request_id': 'req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:24.828960 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aec2f4ac3a1246c7a65da4645d89392b', 'request_id': 'req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:24.837628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aec2f4ac3a1246c7a65da4645d89392b', 'request_id': 'req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:24.851518 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] The token cache is being invalidate because project aec2f4ac3a1246c7a65da4645d89392b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:01:24.851888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:24.852642 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:24.857409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1437/2804] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:01:24 2022] DELETE /identity/v3/projects/aec2f4ac3a1246c7a65da4645d89392b => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:24.859052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:24.861118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:24.875384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3d986-f436-44df-8867-3609ab8c3605 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:24.915811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:24.946744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:24.947332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:24.947635 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:24.947918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] PATH_INFO: `/v3/users/8a5e7674e94e41ea9aecef240ad18954` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:24.949302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a5e7674e94e41ea9aecef240ad18954)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:24.952796 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:25.016960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a5e7674e94e41ea9aecef240ad18954'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:25.026689 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a5e7674e94e41ea9aecef240ad18954'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:25.036287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a5e7674e94e41ea9aecef240ad18954'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:25.053856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a5e7674e94e41ea9aecef240ad18954'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:25.054956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a5e7674e94e41ea9aecef240ad18954'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:25.061768 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1368/2805] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:01:24 2022] DELETE /identity/v3/users/8a5e7674e94e41ea9aecef240ad18954 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:01:25.362912 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:25.364590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:25.384094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef70d7f-db54-49ca-bf7f-38d37a5d3dce admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:25.450078 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:25.456939 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:25.457867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:25.458420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:25.458777 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] PATH_INFO: `/v3/projects/a0da68c294984a7daa71c8b64cb7cdb6` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:25.460452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0da68c294984a7daa71c8b64cb7cdb6)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:25.463554 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:25.517094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0da68c294984a7daa71c8b64cb7cdb6', 'request_id': 'req-e6cf5243-2568-491f-9363-649b73858c2b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:25.527371 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0da68c294984a7daa71c8b64cb7cdb6', 'request_id': 'req-e6cf5243-2568-491f-9363-649b73858c2b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:25.535131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0da68c294984a7daa71c8b64cb7cdb6', 'request_id': 'req-e6cf5243-2568-491f-9363-649b73858c2b'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:25.548888 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] The token cache is being invalidate because project a0da68c294984a7daa71c8b64cb7cdb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:01:25.549313 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:25.550127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e6cf5243-2568-491f-9363-649b73858c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:25.551830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1438/2806] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:01:25 2022] DELETE /identity/v3/projects/a0da68c294984a7daa71c8b64cb7cdb6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:35.058061 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:35.545539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:35.938161 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:35.939762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:35.955068 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb56b16-7b91-42f2-b90b-56f2e06d7ae5 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:36.034286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:36.035842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:36.036337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:36.036843 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] PATH_INFO: `/v3/users/80ce4b7a22f142c9a33cbd3b04e77cf0` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:36.038893 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ce4b7a22f142c9a33cbd3b04e77cf0)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:36.042423 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:36.115882 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ce4b7a22f142c9a33cbd3b04e77cf0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:36.124589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ce4b7a22f142c9a33cbd3b04e77cf0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:36.132627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ce4b7a22f142c9a33cbd3b04e77cf0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:36.146515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ce4b7a22f142c9a33cbd3b04e77cf0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:36.147411 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ce4b7a22f142c9a33cbd3b04e77cf0'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:36.154526 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1369/2807] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:01:35 2022] DELETE /identity/v3/users/80ce4b7a22f142c9a33cbd3b04e77cf0 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:36.159564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc7937-29b9-4a7c-97fe-d113cbd122ca None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:36.159908 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc7937-29b9-4a7c-97fe-d113cbd122ca None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:36.160202 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc7937-29b9-4a7c-97fe-d113cbd122ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:36.180730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-d7dc7937-29b9-4a7c-97fe-d113cbd122ca None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:36.182149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d7dc7937-29b9-4a7c-97fe-d113cbd122ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:36.253593 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1439/2808] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:36.265068 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:36.266707 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:36.341595 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0029ef4-c1ae-4b20-a77f-0bafb702e722 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:36.345960 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd29ed2-ed86-4c5f-b6ed-4d96e4f716c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:36.346633 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd29ed2-ed86-4c5f-b6ed-4d96e4f716c3 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:36.346918 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd29ed2-ed86-4c5f-b6ed-4d96e4f716c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:36.347195 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd29ed2-ed86-4c5f-b6ed-4d96e4f716c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:36.359939 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd29ed2-ed86-4c5f-b6ed-4d96e4f716c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:36.365519 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd29ed2-ed86-4c5f-b6ed-4d96e4f716c3 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:36.377301 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1370/2809] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:36.676901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc7937-29b9-4a7c-97fe-d113cbd122ca None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:36.678417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d7dc7937-29b9-4a7c-97fe-d113cbd122ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:36.694346 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc7937-29b9-4a7c-97fe-d113cbd122ca None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:36.700937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:36.701861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:36.702233 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:36.702421 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] PATH_INFO: `/v3/projects/e24f214e7d5d4d32af0df7d71d616b26` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:36.703830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24f214e7d5d4d32af0df7d71d616b26)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:36.707087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:36.768226 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24f214e7d5d4d32af0df7d71d616b26', 'request_id': 'req-d750e324-b85d-4615-9bdb-b21e8bc2a289'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:36.776162 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24f214e7d5d4d32af0df7d71d616b26', 'request_id': 'req-d750e324-b85d-4615-9bdb-b21e8bc2a289'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:36.781997 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24f214e7d5d4d32af0df7d71d616b26', 'request_id': 'req-d750e324-b85d-4615-9bdb-b21e8bc2a289'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:36.793596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] The token cache is being invalidate because project e24f214e7d5d4d32af0df7d71d616b26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:01:36.793887 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:36.794542 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d750e324-b85d-4615-9bdb-b21e8bc2a289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:01:36.796094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1440/2810] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:01:36 2022] DELETE /identity/v3/projects/e24f214e7d5d4d32af0df7d71d616b26 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:36.801623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a676fb-c34e-4c0b-9621-5a69f5e35c77 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:36.801917 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a676fb-c34e-4c0b-9621-5a69f5e35c77 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:36.802227 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a676fb-c34e-4c0b-9621-5a69f5e35c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:36.821661 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-55a676fb-c34e-4c0b-9621-5a69f5e35c77 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:36.822877 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-55a676fb-c34e-4c0b-9621-5a69f5e35c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:36.840910 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:36.890560 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1371/2811] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:36.898211 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8332286-a0b9-478c-824a-c56919380c38 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:36.898528 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8332286-a0b9-478c-824a-c56919380c38 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:36.898813 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8332286-a0b9-478c-824a-c56919380c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:36.921706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-c8332286-a0b9-478c-824a-c56919380c38 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:36.922950 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c8332286-a0b9-478c-824a-c56919380c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:36.932241 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1441/2812] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:36.937076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a676fb-c34e-4c0b-9621-5a69f5e35c77 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:36.947390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a676fb-c34e-4c0b-9621-5a69f5e35c77 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:36.952217 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd3df6-908f-489b-a3d7-1b2950b4e148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:36.952900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd3df6-908f-489b-a3d7-1b2950b4e148 admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:36.953203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd3df6-908f-489b-a3d7-1b2950b4e148 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:36.953508 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd3df6-908f-489b-a3d7-1b2950b4e148 admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:36.954265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd3df6-908f-489b-a3d7-1b2950b4e148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:01:36.954575 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd3df6-908f-489b-a3d7-1b2950b4e148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:36.956254 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd3df6-908f-489b-a3d7-1b2950b4e148 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:36.970139 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1372/2813] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:01:36.974681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8332286-a0b9-478c-824a-c56919380c38 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:36.985608 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8332286-a0b9-478c-824a-c56919380c38 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:36.992486 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a25a2-fb97-4421-8d81-05149f5d13fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:36.993286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a25a2-fb97-4421-8d81-05149f5d13fd admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:36.993545 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a25a2-fb97-4421-8d81-05149f5d13fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:36.994052 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a25a2-fb97-4421-8d81-05149f5d13fd admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:36.995080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a25a2-fb97-4421-8d81-05149f5d13fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:36.997206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a25a2-fb97-4421-8d81-05149f5d13fd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:37.027003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1442/2814] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:01:36 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:37.032220 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd3df6-908f-489b-a3d7-1b2950b4e148 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:37.044414 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd3df6-908f-489b-a3d7-1b2950b4e148 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:37.108923 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef5ca3-f38d-44e1-9aed-51c18988b05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:37.109625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef5ca3-f38d-44e1-9aed-51c18988b05e admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:37.109919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef5ca3-f38d-44e1-9aed-51c18988b05e admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:37.110193 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef5ca3-f38d-44e1-9aed-51c18988b05e admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:37.111014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef5ca3-f38d-44e1-9aed-51c18988b05e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:37.112940 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef5ca3-f38d-44e1-9aed-51c18988b05e admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:37.133847 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1373/2815] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:01:37 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:01:37.138291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a25a2-fb97-4421-8d81-05149f5d13fd admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:37.149019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a25a2-fb97-4421-8d81-05149f5d13fd admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:37.154875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ee75b-66b9-4c8c-9e2d-0e74fd8b57dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:37.155638 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ee75b-66b9-4c8c-9e2d-0e74fd8b57dd admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:37.155859 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ee75b-66b9-4c8c-9e2d-0e74fd8b57dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:37.156132 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ee75b-66b9-4c8c-9e2d-0e74fd8b57dd admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:37.156889 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ee75b-66b9-4c8c-9e2d-0e74fd8b57dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:37.158525 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ee75b-66b9-4c8c-9e2d-0e74fd8b57dd admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:37.174018 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1443/2816] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:01:37.174806 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef5ca3-f38d-44e1-9aed-51c18988b05e admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:37.185706 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef5ca3-f38d-44e1-9aed-51c18988b05e admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:37.191360 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53a70f-7aed-4e51-96d2-1012489e158c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:37.192031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53a70f-7aed-4e51-96d2-1012489e158c admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:37.192314 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53a70f-7aed-4e51-96d2-1012489e158c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:37.192590 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53a70f-7aed-4e51-96d2-1012489e158c admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:37.193317 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53a70f-7aed-4e51-96d2-1012489e158c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:37.194941 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53a70f-7aed-4e51-96d2-1012489e158c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:37.205555 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1374/2817] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:01:37.209921 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ee75b-66b9-4c8c-9e2d-0e74fd8b57dd admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:37.221253 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ee75b-66b9-4c8c-9e2d-0e74fd8b57dd admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:37.226417 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba559f-2f4d-4abb-9fec-eb61c95dfae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:37.227364 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba559f-2f4d-4abb-9fec-eb61c95dfae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:37.227644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba559f-2f4d-4abb-9fec-eb61c95dfae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:37.227923 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba559f-2f4d-4abb-9fec-eb61c95dfae2 admin admin] PATH_INFO: `/v3/projects/7039c9eea22c434b886c7caebadd94ef/users/61f1dc8cbd6746a698ade3aeddd31cce/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:37.246794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba559f-2f4d-4abb-9fec-eb61c95dfae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7039c9eea22c434b886c7caebadd94ef, user_id=61f1dc8cbd6746a698ade3aeddd31cce, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:37.249773 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba559f-2f4d-4abb-9fec-eb61c95dfae2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:37.264128 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1444/2818] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:01:37 2022] PUT /identity/v3/projects/7039c9eea22c434b886c7caebadd94ef/users/61f1dc8cbd6746a698ade3aeddd31cce/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:01:37.275362 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761d9d3-9c40-4cc6-ab20-bfbb44c6f761 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:37.275650 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761d9d3-9c40-4cc6-ab20-bfbb44c6f761 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:37.275926 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761d9d3-9c40-4cc6-ab20-bfbb44c6f761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:37.304743 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a761d9d3-9c40-4cc6-ab20-bfbb44c6f761 None None] MFA Rules not processed for user `61f1dc8cbd6746a698ade3aeddd31cce`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:37.306014 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a761d9d3-9c40-4cc6-ab20-bfbb44c6f761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:37.379756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1375/2819] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:37.388462 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba559f-2f4d-4abb-9fec-eb61c95dfae2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:37.391367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-49ba559f-2f4d-4abb-9fec-eb61c95dfae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:37.417807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:37.463699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba559f-2f4d-4abb-9fec-eb61c95dfae2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:37.468692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7cfa0-41be-4b35-acd9-92c9e010d32d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:37.469342 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7cfa0-41be-4b35-acd9-92c9e010d32d service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:37.469651 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7cfa0-41be-4b35-acd9-92c9e010d32d service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:37.469928 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7cfa0-41be-4b35-acd9-92c9e010d32d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:37.481084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7cfa0-41be-4b35-acd9-92c9e010d32d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:37.482923 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7cfa0-41be-4b35-acd9-92c9e010d32d service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:37.492565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1445/2820] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:44.850419 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63416cdb-da8e-45ae-aae2-be636de7cbe0 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:44.850998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63416cdb-da8e-45ae-aae2-be636de7cbe0 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:44.850998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63416cdb-da8e-45ae-aae2-be636de7cbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:44.869168 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-63416cdb-da8e-45ae-aae2-be636de7cbe0 None None] MFA Rules not processed for user `cf1d05ee73d4414eb51e2a7be838e28f`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:44.870514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-63416cdb-da8e-45ae-aae2-be636de7cbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:44.882966 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:44.930346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1376/2821] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:01:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:45.286403 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7cfa0-41be-4b35-acd9-92c9e010d32d service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:45.287856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-7eb7cfa0-41be-4b35-acd9-92c9e010d32d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:45.370512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7cfa0-41be-4b35-acd9-92c9e010d32d service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:45.376261 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b4feb-112e-47b2-964f-d9b7a7f45cb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:45.377352 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b4feb-112e-47b2-964f-d9b7a7f45cb6 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:45.377868 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b4feb-112e-47b2-964f-d9b7a7f45cb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:45.378351 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b4feb-112e-47b2-964f-d9b7a7f45cb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:45.381062 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c38b4feb-112e-47b2-964f-d9b7a7f45cb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:45.392565 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b4feb-112e-47b2-964f-d9b7a7f45cb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:45.394520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b4feb-112e-47b2-964f-d9b7a7f45cb6 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:45.405594 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1446/2822] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:45.872926 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63416cdb-da8e-45ae-aae2-be636de7cbe0 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:45.875176 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-63416cdb-da8e-45ae-aae2-be636de7cbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:45.959685 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63416cdb-da8e-45ae-aae2-be636de7cbe0 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:45.965001 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf31a6c-9bc3-4248-9c2c-7ffea444e786 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:45.965695 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf31a6c-9bc3-4248-9c2c-7ffea444e786 service placement] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:45.965975 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf31a6c-9bc3-4248-9c2c-7ffea444e786 service placement] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:45.966250 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf31a6c-9bc3-4248-9c2c-7ffea444e786 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:45.968072 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-9bf31a6c-9bc3-4248-9c2c-7ffea444e786 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:45.980581 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf31a6c-9bc3-4248-9c2c-7ffea444e786 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:45.982199 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf31a6c-9bc3-4248-9c2c-7ffea444e786 service placement] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:45.992849 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1377/2823] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:55.399681 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=43 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:55.988200 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=49 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:02:03.092823 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b4feb-112e-47b2-964f-d9b7a7f45cb6 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:03.094795 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-c38b4feb-112e-47b2-964f-d9b7a7f45cb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:03.220027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b4feb-112e-47b2-964f-d9b7a7f45cb6 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:03.230472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f12c0-0784-45d7-97d4-6a106c313cfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:03.231587 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f12c0-0784-45d7-97d4-6a106c313cfd service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:03.232078 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f12c0-0784-45d7-97d4-6a106c313cfd service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:03.232543 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f12c0-0784-45d7-97d4-6a106c313cfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:03.235718 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-007f12c0-0784-45d7-97d4-6a106c313cfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:03.255099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f12c0-0784-45d7-97d4-6a106c313cfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:03.256732 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f12c0-0784-45d7-97d4-6a106c313cfd service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:03.270183 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1447/2824] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:02:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:02:05.462970 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf31a6c-9bc3-4248-9c2c-7ffea444e786 service placement] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:05.473536 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf31a6c-9bc3-4248-9c2c-7ffea444e786 service placement] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:05.477141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16f4ae-82ec-45f9-b624-662380472bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:05.477855 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16f4ae-82ec-45f9-b624-662380472bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:05.478136 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16f4ae-82ec-45f9-b624-662380472bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:05.478416 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16f4ae-82ec-45f9-b624-662380472bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:05.481331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ef16f4ae-82ec-45f9-b624-662380472bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:05.574927 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16f4ae-82ec-45f9-b624-662380472bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:05.577098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16f4ae-82ec-45f9-b624-662380472bc8 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:05.591158 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1378/2825] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:02:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:02:13.263954 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=34 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:02:15.585638 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=30 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:02:21.966160 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f12c0-0784-45d7-97d4-6a106c313cfd service nova] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:22.043820 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f12c0-0784-45d7-97d4-6a106c313cfd service nova] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:22.048828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:22.050131 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:22.050269 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:22.050484 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] PATH_INFO: `/v3/users/61f1dc8cbd6746a698ade3aeddd31cce` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:22.051826 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f1dc8cbd6746a698ade3aeddd31cce)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:22.054285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:22.107897 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f1dc8cbd6746a698ade3aeddd31cce'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:02:22.121067 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f1dc8cbd6746a698ade3aeddd31cce'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:02:22.129557 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f1dc8cbd6746a698ade3aeddd31cce'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:02:22.130032 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f1dc8cbd6746a698ade3aeddd31cce'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:02:22.138177 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f1dc8cbd6746a698ade3aeddd31cce'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:02:22.144362 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1448/2826] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:02:21 2022] DELETE /identity/v3/users/61f1dc8cbd6746a698ade3aeddd31cce => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:02:22.152130 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41e350-f2d9-4f42-9d73-c0131d181997 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:22.152697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41e350-f2d9-4f42-9d73-c0131d181997 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:22.152828 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41e350-f2d9-4f42-9d73-c0131d181997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:22.174157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-3f41e350-f2d9-4f42-9d73-c0131d181997 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:02:22.176063 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3f41e350-f2d9-4f42-9d73-c0131d181997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:22.252984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1379/2827] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:02:22.266355 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:22.268680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:22.335189 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d03215-64ce-4ea2-9f68-d5a37fdc5a36 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:22.338850 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454fd46-6706-439f-a8c0-d5061cdc54cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:22.339519 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454fd46-6706-439f-a8c0-d5061cdc54cf service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:22.339811 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454fd46-6706-439f-a8c0-d5061cdc54cf service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:22.340095 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454fd46-6706-439f-a8c0-d5061cdc54cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:22.350395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454fd46-6706-439f-a8c0-d5061cdc54cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:22.352003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454fd46-6706-439f-a8c0-d5061cdc54cf service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:22.362666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1449/2828] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:02:22.690277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41e350-f2d9-4f42-9d73-c0131d181997 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:22.692341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-3f41e350-f2d9-4f42-9d73-c0131d181997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:22.706278 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41e350-f2d9-4f42-9d73-c0131d181997 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:22.711390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:22.712375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:22.712644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:22.712833 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] PATH_INFO: `/v3/projects/7039c9eea22c434b886c7caebadd94ef` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:22.714089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7039c9eea22c434b886c7caebadd94ef)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:22.715994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:22.780631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7039c9eea22c434b886c7caebadd94ef', 'request_id': 'req-7cb56818-3a9d-4b70-8a51-a116df71741f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:02:22.787346 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7039c9eea22c434b886c7caebadd94ef', 'request_id': 'req-7cb56818-3a9d-4b70-8a51-a116df71741f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:02:22.793336 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7039c9eea22c434b886c7caebadd94ef', 'request_id': 'req-7cb56818-3a9d-4b70-8a51-a116df71741f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:02:22.807497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] The token cache is being invalidate because project 7039c9eea22c434b886c7caebadd94ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:02:22.807781 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:02:22.808456 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-7cb56818-3a9d-4b70-8a51-a116df71741f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:02:22.810052 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1380/2829] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:02:22 2022] DELETE /identity/v3/projects/7039c9eea22c434b886c7caebadd94ef => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:02:22.815024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a587f-ddbe-46d0-85e4-d24aaca65bcd None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:22.815321 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a587f-ddbe-46d0-85e4-d24aaca65bcd None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:22.815590 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a587f-ddbe-46d0-85e4-d24aaca65bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:22.836134 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-935a587f-ddbe-46d0-85e4-d24aaca65bcd None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:02:22.837599 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-935a587f-ddbe-46d0-85e4-d24aaca65bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:22.855919 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:02:22.906738 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1450/2830] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:02:22.915003 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3553b-3b6e-4e3e-8083-a9fb8040c78f None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:22.915294 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3553b-3b6e-4e3e-8083-a9fb8040c78f None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:22.915864 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3553b-3b6e-4e3e-8083-a9fb8040c78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:22.936073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-a6d3553b-3b6e-4e3e-8083-a9fb8040c78f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:02:22.937332 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-a6d3553b-3b6e-4e3e-8083-a9fb8040c78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:22.947561 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1381/2831] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:02:22.953466 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a587f-ddbe-46d0-85e4-d24aaca65bcd None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:22.970319 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a587f-ddbe-46d0-85e4-d24aaca65bcd None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:22.975392 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dee42-3c44-40ca-942e-75fccbccc44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:22.976070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dee42-3c44-40ca-942e-75fccbccc44c admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:22.976359 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dee42-3c44-40ca-942e-75fccbccc44c admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:22.976637 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1dee42-3c44-40ca-942e-75fccbccc44c admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:22.977406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1dee42-3c44-40ca-942e-75fccbccc44c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:02:22.977721 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1dee42-3c44-40ca-942e-75fccbccc44c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:22.979918 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1dee42-3c44-40ca-942e-75fccbccc44c admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:22.999901 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1451/2832] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:02:23.007073 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3553b-3b6e-4e3e-8083-a9fb8040c78f None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:23.022173 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3553b-3b6e-4e3e-8083-a9fb8040c78f None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:23.027374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d0003-2e0d-4d1f-8961-fb0ec4de1cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:23.028051 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d0003-2e0d-4d1f-8961-fb0ec4de1cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:23.028340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d0003-2e0d-4d1f-8961-fb0ec4de1cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:23.028616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d0003-2e0d-4d1f-8961-fb0ec4de1cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:23.029323 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d0003-2e0d-4d1f-8961-fb0ec4de1cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:23.031287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d0003-2e0d-4d1f-8961-fb0ec4de1cb2 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:23.061380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1382/2833] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:02:23 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:02:23.066307 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dee42-3c44-40ca-942e-75fccbccc44c admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:23.080979 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1dee42-3c44-40ca-942e-75fccbccc44c admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:23.149146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aa36b-c30b-4ea4-a737-219efc8adf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:23.149895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aa36b-c30b-4ea4-a737-219efc8adf17 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:23.150101 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aa36b-c30b-4ea4-a737-219efc8adf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:23.150384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aa36b-c30b-4ea4-a737-219efc8adf17 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:23.151075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7aa36b-c30b-4ea4-a737-219efc8adf17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:23.152999 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7aa36b-c30b-4ea4-a737-219efc8adf17 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:23.175332 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1452/2834] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:02:23 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:02:23.179693 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d0003-2e0d-4d1f-8961-fb0ec4de1cb2 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:23.191287 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d0003-2e0d-4d1f-8961-fb0ec4de1cb2 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:23.196385 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a23f3d-23e6-4c85-80c8-16c1cb643dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:23.197137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a23f3d-23e6-4c85-80c8-16c1cb643dff admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:23.197472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a23f3d-23e6-4c85-80c8-16c1cb643dff admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:23.197745 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a23f3d-23e6-4c85-80c8-16c1cb643dff admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:23.198479 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a23f3d-23e6-4c85-80c8-16c1cb643dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:23.200196 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a23f3d-23e6-4c85-80c8-16c1cb643dff admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:23.214428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1383/2835] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:02:23.218461 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aa36b-c30b-4ea4-a737-219efc8adf17 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:23.229628 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aa36b-c30b-4ea4-a737-219efc8adf17 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:23.234451 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca425a6f-1b6e-4eaa-969a-ff8bc569fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:23.235113 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca425a6f-1b6e-4eaa-969a-ff8bc569fb07 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:23.235426 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca425a6f-1b6e-4eaa-969a-ff8bc569fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:23.235706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca425a6f-1b6e-4eaa-969a-ff8bc569fb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:23.236435 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca425a6f-1b6e-4eaa-969a-ff8bc569fb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:23.238071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca425a6f-1b6e-4eaa-969a-ff8bc569fb07 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:23.249048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1453/2836] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:02:23.253055 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a23f3d-23e6-4c85-80c8-16c1cb643dff admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:23.263307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a23f3d-23e6-4c85-80c8-16c1cb643dff admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:23.267994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30a7b1-05d5-45e3-84cb-4902dfdc6aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:23.268864 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30a7b1-05d5-45e3-84cb-4902dfdc6aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:23.269141 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30a7b1-05d5-45e3-84cb-4902dfdc6aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:23.269450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30a7b1-05d5-45e3-84cb-4902dfdc6aa7 admin admin] PATH_INFO: `/v3/projects/44d7d00a1ff0437cb7fc46658c3a1f79/users/a02317457dae42069b86da4bd35278b4/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:23.285053 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30a7b1-05d5-45e3-84cb-4902dfdc6aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44d7d00a1ff0437cb7fc46658c3a1f79, user_id=a02317457dae42069b86da4bd35278b4, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:23.287717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30a7b1-05d5-45e3-84cb-4902dfdc6aa7 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:23.301329 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1384/2837] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:02:23 2022] PUT /identity/v3/projects/44d7d00a1ff0437cb7fc46658c3a1f79/users/a02317457dae42069b86da4bd35278b4/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:02:23.307397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460346f-5a8d-45e0-8150-5b4cca27a340 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:23.307688 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460346f-5a8d-45e0-8150-5b4cca27a340 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:23.307968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460346f-5a8d-45e0-8150-5b4cca27a340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:23.326665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-2460346f-5a8d-45e0-8150-5b4cca27a340 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:02:23.327874 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2460346f-5a8d-45e0-8150-5b4cca27a340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:23.393049 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1454/2838] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:02:23.405951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30a7b1-05d5-45e3-84cb-4902dfdc6aa7 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:23.405951 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-4e30a7b1-05d5-45e3-84cb-4902dfdc6aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:23.428497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:02:23.466775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30a7b1-05d5-45e3-84cb-4902dfdc6aa7 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:23.470811 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237eef8-78b3-4edc-8192-c2ec98f0a969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:23.471459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237eef8-78b3-4edc-8192-c2ec98f0a969 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:23.471744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237eef8-78b3-4edc-8192-c2ec98f0a969 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:23.472089 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237eef8-78b3-4edc-8192-c2ec98f0a969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:23.482286 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237eef8-78b3-4edc-8192-c2ec98f0a969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:23.483754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237eef8-78b3-4edc-8192-c2ec98f0a969 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:23.493762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1385/2839] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:02:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:02:24.199679 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc3fe5-a88b-4f67-9460-2c362073ec5e None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:24.200285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc3fe5-a88b-4f67-9460-2c362073ec5e None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:24.200285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc3fe5-a88b-4f67-9460-2c362073ec5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:24.234165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-9edc3fe5-a88b-4f67-9460-2c362073ec5e None None] MFA Rules not processed for user `a02317457dae42069b86da4bd35278b4`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:02:24.235873 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9edc3fe5-a88b-4f67-9460-2c362073ec5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:24.262880 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:02:24.315808 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1455/2840] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:02:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:02:24.330094 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237eef8-78b3-4edc-8192-c2ec98f0a969 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:24.331535 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5237eef8-78b3-4edc-8192-c2ec98f0a969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:24.393038 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237eef8-78b3-4edc-8192-c2ec98f0a969 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:24.396775 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc291dc-88cd-41a0-8f56-12e32355b787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:24.397462 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc291dc-88cd-41a0-8f56-12e32355b787 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:24.397744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc291dc-88cd-41a0-8f56-12e32355b787 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:24.398013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc291dc-88cd-41a0-8f56-12e32355b787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:24.406524 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc291dc-88cd-41a0-8f56-12e32355b787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:24.407935 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc291dc-88cd-41a0-8f56-12e32355b787 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:24.416980 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1386/2841] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:02:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:02:25.831121 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911a48-f947-4ad2-9239-5adca3c0c530 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:25.831719 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911a48-f947-4ad2-9239-5adca3c0c530 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:25.831947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911a48-f947-4ad2-9239-5adca3c0c530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:25.861124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-a3911a48-f947-4ad2-9239-5adca3c0c530 None None] MFA Rules not processed for user `a02317457dae42069b86da4bd35278b4`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:02:25.862736 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a3911a48-f947-4ad2-9239-5adca3c0c530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:25.870536 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1456/2842] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:02:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:02:34.412787 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=43 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:02:35.852747 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=22 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:11.681755 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc291dc-88cd-41a0-8f56-12e32355b787 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:11.683277 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5dc291dc-88cd-41a0-8f56-12e32355b787 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:11.766718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc291dc-88cd-41a0-8f56-12e32355b787 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:11.771822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3943f3-8625-4c6e-83ea-a79280cf0d5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:11.772472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3943f3-8625-4c6e-83ea-a79280cf0d5c service placement] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:11.772758 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3943f3-8625-4c6e-83ea-a79280cf0d5c service placement] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:11.773036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3943f3-8625-4c6e-83ea-a79280cf0d5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:11.774904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5b3943f3-8625-4c6e-83ea-a79280cf0d5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:11.787699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3943f3-8625-4c6e-83ea-a79280cf0d5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:11.789291 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3943f3-8625-4c6e-83ea-a79280cf0d5c service placement] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:11.799905 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1387/2843] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:21.795295 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=32 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:41.061922 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f280819-5b17-408a-b020-54ed0da2f8ac None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:41.062513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f280819-5b17-408a-b020-54ed0da2f8ac None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:41.062513 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f280819-5b17-408a-b020-54ed0da2f8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:41.080591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-6f280819-5b17-408a-b020-54ed0da2f8ac None None] MFA Rules not processed for user `a02317457dae42069b86da4bd35278b4`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:41.081851 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6f280819-5b17-408a-b020-54ed0da2f8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:41.088870 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1457/2844] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:03:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:41.336354 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3943f3-8625-4c6e-83ea-a79280cf0d5c service placement] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:41.337805 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-5b3943f3-8625-4c6e-83ea-a79280cf0d5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:41.351232 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3943f3-8625-4c6e-83ea-a79280cf0d5c service placement] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:41.355132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bda138-7cf0-4fc2-a094-2cf695e3e249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:41.355825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bda138-7cf0-4fc2-a094-2cf695e3e249 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:41.356105 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bda138-7cf0-4fc2-a094-2cf695e3e249 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:41.356382 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bda138-7cf0-4fc2-a094-2cf695e3e249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:41.358634 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-86bda138-7cf0-4fc2-a094-2cf695e3e249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:41.370718 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bda138-7cf0-4fc2-a094-2cf695e3e249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:41.372172 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bda138-7cf0-4fc2-a094-2cf695e3e249 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:41.381754 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1388/2845] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:51.075165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=3 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:51.377770 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=6 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:14.175804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f280819-5b17-408a-b020-54ed0da2f8ac None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:14.185662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f280819-5b17-408a-b020-54ed0da2f8ac None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:14.189946 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9643268-05c6-4fe6-9689-6cc08f67d5f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:14.190591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9643268-05c6-4fe6-9689-6cc08f67d5f6 service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:14.190895 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9643268-05c6-4fe6-9689-6cc08f67d5f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:14.191151 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9643268-05c6-4fe6-9689-6cc08f67d5f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:14.192984 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-f9643268-05c6-4fe6-9689-6cc08f67d5f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:14.266507 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9643268-05c6-4fe6-9689-6cc08f67d5f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:14.267937 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9643268-05c6-4fe6-9689-6cc08f67d5f6 service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:14.279156 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1458/2846] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:04:16.882777 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bda138-7cf0-4fc2-a094-2cf695e3e249 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:16.891956 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bda138-7cf0-4fc2-a094-2cf695e3e249 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:16.896769 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd3741-3c8c-452f-8f1d-ca356ec26723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:16.897436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd3741-3c8c-452f-8f1d-ca356ec26723 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:16.897766 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd3741-3c8c-452f-8f1d-ca356ec26723 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:16.898000 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd3741-3c8c-452f-8f1d-ca356ec26723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:16.899839 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-31bd3741-3c8c-452f-8f1d-ca356ec26723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:16.913589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd3741-3c8c-452f-8f1d-ca356ec26723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:16.915018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd3741-3c8c-452f-8f1d-ca356ec26723 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:16.926060 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1389/2847] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:04:24.274127 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=34 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:26.921110 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=6 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:44.199986 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9643268-05c6-4fe6-9689-6cc08f67d5f6 service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:44.213242 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9643268-05c6-4fe6-9689-6cc08f67d5f6 service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:44.218007 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ac52b-97c9-4e64-a1ff-997c571293f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:44.218935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ac52b-97c9-4e64-a1ff-997c571293f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:44.219472 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ac52b-97c9-4e64-a1ff-997c571293f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:44.219752 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ac52b-97c9-4e64-a1ff-997c571293f2 admin admin] PATH_INFO: `/v3/users/a02317457dae42069b86da4bd35278b4` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:44.221003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ac52b-97c9-4e64-a1ff-997c571293f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02317457dae42069b86da4bd35278b4)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:44.222967 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ac52b-97c9-4e64-a1ff-997c571293f2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:44.287219 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a51ac52b-97c9-4e64-a1ff-997c571293f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02317457dae42069b86da4bd35278b4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:44.301287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a51ac52b-97c9-4e64-a1ff-997c571293f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02317457dae42069b86da4bd35278b4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:44.311324 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a51ac52b-97c9-4e64-a1ff-997c571293f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02317457dae42069b86da4bd35278b4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:44.312043 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a51ac52b-97c9-4e64-a1ff-997c571293f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02317457dae42069b86da4bd35278b4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:44.321197 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a51ac52b-97c9-4e64-a1ff-997c571293f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02317457dae42069b86da4bd35278b4'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:44.327692 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1459/2848] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:04:44 2022] DELETE /identity/v3/users/a02317457dae42069b86da4bd35278b4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:04:44.577789 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd3741-3c8c-452f-8f1d-ca356ec26723 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:44.589133 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd3741-3c8c-452f-8f1d-ca356ec26723 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:44.656458 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:44.657313 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:44.657631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:44.657904 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] PATH_INFO: `/v3/projects/44d7d00a1ff0437cb7fc46658c3a1f79` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:44.659228 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] RBAC: Authorizing `identity:delete_project(project_id=44d7d00a1ff0437cb7fc46658c3a1f79)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:44.661976 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:44.716127 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44d7d00a1ff0437cb7fc46658c3a1f79', 'request_id': 'req-a7298936-30c2-41e2-8bea-19480e7d146f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:44.722807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44d7d00a1ff0437cb7fc46658c3a1f79', 'request_id': 'req-a7298936-30c2-41e2-8bea-19480e7d146f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:44.731026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44d7d00a1ff0437cb7fc46658c3a1f79', 'request_id': 'req-a7298936-30c2-41e2-8bea-19480e7d146f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:44.746642 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] The token cache is being invalidate because project 44d7d00a1ff0437cb7fc46658c3a1f79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92311) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:04:44.746931 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:44.748009 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-a7298936-30c2-41e2-8bea-19480e7d146f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:44.749751 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1390/2849] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:04:44 2022] DELETE /identity/v3/projects/44d7d00a1ff0437cb7fc46658c3a1f79 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:04:44.755805 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e879cac-6f89-413d-9899-2d887642ff1f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:44.756200 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e879cac-6f89-413d-9899-2d887642ff1f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:44.756446 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e879cac-6f89-413d-9899-2d887642ff1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:44.777059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-3e879cac-6f89-413d-9899-2d887642ff1f None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:44.778362 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-3e879cac-6f89-413d-9899-2d887642ff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:44.855680 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1460/2850] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:44.864824 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd634e4-c32c-4b3c-8acf-1321acdd7c33 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:44.865105 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd634e4-c32c-4b3c-8acf-1321acdd7c33 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:44.865409 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd634e4-c32c-4b3c-8acf-1321acdd7c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:44.891492 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-1cd634e4-c32c-4b3c-8acf-1321acdd7c33 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:44.893310 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-1cd634e4-c32c-4b3c-8acf-1321acdd7c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:44.901725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1391/2851] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:44.907622 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e879cac-6f89-413d-9899-2d887642ff1f None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:44.919395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e879cac-6f89-413d-9899-2d887642ff1f None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:44.924074 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef7c31-d286-4650-ae98-4362b6d5b191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:44.924770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef7c31-d286-4650-ae98-4362b6d5b191 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:44.925056 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef7c31-d286-4650-ae98-4362b6d5b191 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:44.925338 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef7c31-d286-4650-ae98-4362b6d5b191 admin admin] PATH_INFO: `/v3/domains` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:44.926099 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef7c31-d286-4650-ae98-4362b6d5b191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92310) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:04:44.926409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef7c31-d286-4650-ae98-4362b6d5b191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:44.928500 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef7c31-d286-4650-ae98-4362b6d5b191 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:44.941016 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1461/2852] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:04:44.948493 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd634e4-c32c-4b3c-8acf-1321acdd7c33 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:44.960340 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd634e4-c32c-4b3c-8acf-1321acdd7c33 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:44.965522 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ed6a7-a9d6-40da-936d-0a344bc37481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:44.966187 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ed6a7-a9d6-40da-936d-0a344bc37481 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:44.966475 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ed6a7-a9d6-40da-936d-0a344bc37481 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:44.966778 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ed6a7-a9d6-40da-936d-0a344bc37481 admin admin] PATH_INFO: `/v3/projects` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:44.967529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ed6a7-a9d6-40da-936d-0a344bc37481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:44.969773 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ed6a7-a9d6-40da-936d-0a344bc37481 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:44.998835 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1392/2853] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:04:44 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:04:45.003283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef7c31-d286-4650-ae98-4362b6d5b191 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:45.013862 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef7c31-d286-4650-ae98-4362b6d5b191 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:45.080278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e62a9-c3bb-40a7-9906-c7863482bac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:45.081003 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e62a9-c3bb-40a7-9906-c7863482bac7 admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:45.081409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e62a9-c3bb-40a7-9906-c7863482bac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:45.081700 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e62a9-c3bb-40a7-9906-c7863482bac7 admin admin] PATH_INFO: `/v3/users` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:45.082441 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e62a9-c3bb-40a7-9906-c7863482bac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:45.085002 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e62a9-c3bb-40a7-9906-c7863482bac7 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:45.092041 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:45.113730 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1462/2854] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:04:45 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:04:45.118446 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ed6a7-a9d6-40da-936d-0a344bc37481 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:45.129625 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ed6a7-a9d6-40da-936d-0a344bc37481 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:45.135300 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a0160-e1c5-4547-893a-ae20788df47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:45.136132 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a0160-e1c5-4547-893a-ae20788df47c admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:45.136380 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a0160-e1c5-4547-893a-ae20788df47c admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:45.136659 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a0160-e1c5-4547-893a-ae20788df47c admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:45.137423 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a0160-e1c5-4547-893a-ae20788df47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:45.139163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a0160-e1c5-4547-893a-ae20788df47c admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:45.154098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1393/2855] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:04:45.158537 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e62a9-c3bb-40a7-9906-c7863482bac7 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:45.169103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e62a9-c3bb-40a7-9906-c7863482bac7 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:45.174185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c85504-c76d-4ff5-8c29-adea5ae5becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:45.174807 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85504-c76d-4ff5-8c29-adea5ae5becc admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:45.175147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85504-c76d-4ff5-8c29-adea5ae5becc admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:45.175723 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85504-c76d-4ff5-8c29-adea5ae5becc admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:45.176193 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c85504-c76d-4ff5-8c29-adea5ae5becc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:45.177856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c85504-c76d-4ff5-8c29-adea5ae5becc admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:45.188530 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1463/2856] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:04:45.193512 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a0160-e1c5-4547-893a-ae20788df47c admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:45.204113 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a0160-e1c5-4547-893a-ae20788df47c admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:45.210056 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb99e0-fa30-4148-98e8-e0f51114c1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:45.210954 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb99e0-fa30-4148-98e8-e0f51114c1c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:45.211241 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb99e0-fa30-4148-98e8-e0f51114c1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:45.211523 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb99e0-fa30-4148-98e8-e0f51114c1c1 admin admin] PATH_INFO: `/v3/projects/590cebe6b5884c81a0f25a5a61145e9c/users/7a68d3cd931b424fb538fdcc3448cd90/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:45.228666 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb99e0-fa30-4148-98e8-e0f51114c1c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=590cebe6b5884c81a0f25a5a61145e9c, user_id=7a68d3cd931b424fb538fdcc3448cd90, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:45.231478 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb99e0-fa30-4148-98e8-e0f51114c1c1 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:45.248276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1394/2857] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:04:45 2022] PUT /identity/v3/projects/590cebe6b5884c81a0f25a5a61145e9c/users/7a68d3cd931b424fb538fdcc3448cd90/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:04:45.260298 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4019b-05b2-454e-8917-84cb42892af2 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:45.261159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4019b-05b2-454e-8917-84cb42892af2 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:45.261159 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4019b-05b2-454e-8917-84cb42892af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:45.292896 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-42d4019b-05b2-454e-8917-84cb42892af2 None None] MFA Rules not processed for user `7a68d3cd931b424fb538fdcc3448cd90`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:45.294254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-42d4019b-05b2-454e-8917-84cb42892af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:45.371647 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1464/2858] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:45.382079 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb99e0-fa30-4148-98e8-e0f51114c1c1 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:45.383822 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-f9cb99e0-fa30-4148-98e8-e0f51114c1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:45.422030 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:45.455878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb99e0-fa30-4148-98e8-e0f51114c1c1 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:45.461950 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24b7bc-19f9-4981-b529-9ebcdd219418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:45.462929 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24b7bc-19f9-4981-b529-9ebcdd219418 service cinder] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:45.463384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24b7bc-19f9-4981-b529-9ebcdd219418 service cinder] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:45.463627 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24b7bc-19f9-4981-b529-9ebcdd219418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:45.476626 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24b7bc-19f9-4981-b529-9ebcdd219418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:45.478431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24b7bc-19f9-4981-b529-9ebcdd219418 service cinder] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:45.488883 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1395/2859] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:04:52.191419 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7516c-8747-4eb9-9fa9-3d17b2a3393f None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:52.191994 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7516c-8747-4eb9-9fa9-3d17b2a3393f None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:52.192190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7516c-8747-4eb9-9fa9-3d17b2a3393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:52.216580 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-9fc7516c-8747-4eb9-9fa9-3d17b2a3393f None None] MFA Rules not processed for user `cf1d05ee73d4414eb51e2a7be838e28f`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:52.217876 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-9fc7516c-8747-4eb9-9fa9-3d17b2a3393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:52.298558 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1465/2860] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:04:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:54.672871 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24b7bc-19f9-4981-b529-9ebcdd219418 service cinder] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:54.743587 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24b7bc-19f9-4981-b529-9ebcdd219418 service cinder] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:54.753293 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:54.754504 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:54.755430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:54.756631 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] PATH_INFO: `/v3/users/7a68d3cd931b424fb538fdcc3448cd90` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:54.757760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a68d3cd931b424fb538fdcc3448cd90)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:54.761011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:54.845687 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a68d3cd931b424fb538fdcc3448cd90'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:54.856236 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a68d3cd931b424fb538fdcc3448cd90'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:54.865531 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a68d3cd931b424fb538fdcc3448cd90'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:54.882466 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a68d3cd931b424fb538fdcc3448cd90'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:54.883761 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a68d3cd931b424fb538fdcc3448cd90'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:54.890762 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1396/2861] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:04:54 2022] DELETE /identity/v3/users/7a68d3cd931b424fb538fdcc3448cd90 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:04:54.895841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92518b21-1446-4718-ae2f-286e09311b93 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:54.896398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92518b21-1446-4718-ae2f-286e09311b93 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:54.896556 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92518b21-1446-4718-ae2f-286e09311b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:54.917104 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-92518b21-1446-4718-ae2f-286e09311b93 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:54.918629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-92518b21-1446-4718-ae2f-286e09311b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:54.992008 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1466/2862] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:55.002468 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:55.004304 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:55.082377 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:55.093877 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1981f-4fc0-45b4-8fd2-82f6e9f0aaa3 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:55.102231 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11249d-4490-455a-914f-a39ac40c90ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:55.102899 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11249d-4490-455a-914f-a39ac40c90ed service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:55.103188 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11249d-4490-455a-914f-a39ac40c90ed service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:55.103470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11249d-4490-455a-914f-a39ac40c90ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:55.113735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11249d-4490-455a-914f-a39ac40c90ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:55.116067 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11249d-4490-455a-914f-a39ac40c90ed service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:55.126738 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1397/2863] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:04:55.581603 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92518b21-1446-4718-ae2f-286e09311b93 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:55.587793 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-92518b21-1446-4718-ae2f-286e09311b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:55.608175 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92518b21-1446-4718-ae2f-286e09311b93 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:55.633089 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:55.634220 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:55.634613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:55.635111 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] PATH_INFO: `/v3/projects/590cebe6b5884c81a0f25a5a61145e9c` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:55.641877 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=590cebe6b5884c81a0f25a5a61145e9c)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:55.645070 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:55.668291 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:55.761566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '590cebe6b5884c81a0f25a5a61145e9c', 'request_id': 'req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:55.774940 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '590cebe6b5884c81a0f25a5a61145e9c', 'request_id': 'req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:55.786305 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '590cebe6b5884c81a0f25a5a61145e9c', 'request_id': 'req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:55.801529 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] The token cache is being invalidate because project 590cebe6b5884c81a0f25a5a61145e9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:04:55.801992 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:55.802783 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-b08e93c3-9a62-44c2-9f55-c1b6142d46ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:04:55.805185 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1467/2864] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:04:55 2022] DELETE /identity/v3/projects/590cebe6b5884c81a0f25a5a61145e9c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:04:55.810155 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb09b0-62e1-496e-bae3-b992ce5e2700 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:55.811057 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb09b0-62e1-496e-bae3-b992ce5e2700 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:55.811573 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb09b0-62e1-496e-bae3-b992ce5e2700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:55.833985 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-afeb09b0-62e1-496e-bae3-b992ce5e2700 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:55.835583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-afeb09b0-62e1-496e-bae3-b992ce5e2700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:55.933740 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1398/2865] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:55.937044 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427e6a1-9ace-42ed-b398-f78450ac76af None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:55.937898 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427e6a1-9ace-42ed-b398-f78450ac76af None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:55.938084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427e6a1-9ace-42ed-b398-f78450ac76af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:55.964601 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-2427e6a1-9ace-42ed-b398-f78450ac76af None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:55.966006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-2427e6a1-9ace-42ed-b398-f78450ac76af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:55.975539 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1468/2866] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:55.980552 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb09b0-62e1-496e-bae3-b992ce5e2700 None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:55.991803 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb09b0-62e1-496e-bae3-b992ce5e2700 None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:55.996512 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41350878-9eab-400a-80aa-32fd696076aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:55.997197 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41350878-9eab-400a-80aa-32fd696076aa admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:55.997514 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41350878-9eab-400a-80aa-32fd696076aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:55.997793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41350878-9eab-400a-80aa-32fd696076aa admin admin] PATH_INFO: `/v3/domains` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:55.998521 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41350878-9eab-400a-80aa-32fd696076aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92311) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:04:55.998854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41350878-9eab-400a-80aa-32fd696076aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:56.000447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41350878-9eab-400a-80aa-32fd696076aa admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:56.019744 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1399/2867] 10.222.0.70 () {58 vars in 1152 bytes} [Tue Jul 19 21:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:04:56.020845 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427e6a1-9ace-42ed-b398-f78450ac76af None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:56.034990 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427e6a1-9ace-42ed-b398-f78450ac76af None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:56.041490 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e17941-f7dc-4d81-8f7f-d8bc474fe6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:56.042165 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e17941-f7dc-4d81-8f7f-d8bc474fe6cc admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:56.042329 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e17941-f7dc-4d81-8f7f-d8bc474fe6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:56.042610 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e17941-f7dc-4d81-8f7f-d8bc474fe6cc admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:56.043310 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e17941-f7dc-4d81-8f7f-d8bc474fe6cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:56.045770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e17941-f7dc-4d81-8f7f-d8bc474fe6cc admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:56.083495 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1469/2868] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:04:56 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:04:56.087619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41350878-9eab-400a-80aa-32fd696076aa admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:56.099856 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41350878-9eab-400a-80aa-32fd696076aa admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:56.167364 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2299461-7460-4c28-be1a-b71fd43c205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:56.168064 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2299461-7460-4c28-be1a-b71fd43c205f admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:56.168337 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2299461-7460-4c28-be1a-b71fd43c205f admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:56.168619 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2299461-7460-4c28-be1a-b71fd43c205f admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:56.169316 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2299461-7460-4c28-be1a-b71fd43c205f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:56.171362 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2299461-7460-4c28-be1a-b71fd43c205f admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:56.194765 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1400/2869] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:04:56 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:04:56.200244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e17941-f7dc-4d81-8f7f-d8bc474fe6cc admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:56.212006 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e17941-f7dc-4d81-8f7f-d8bc474fe6cc admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:56.217140 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9384d05-9259-4b4c-a07e-283b3b940ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:56.217844 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9384d05-9259-4b4c-a07e-283b3b940ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:56.218129 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9384d05-9259-4b4c-a07e-283b3b940ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:56.218410 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9384d05-9259-4b4c-a07e-283b3b940ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:56.219399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9384d05-9259-4b4c-a07e-283b3b940ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:56.220993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9384d05-9259-4b4c-a07e-283b3b940ec2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:56.232395 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1470/2870] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:04:56.236807 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2299461-7460-4c28-be1a-b71fd43c205f admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:56.248070 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2299461-7460-4c28-be1a-b71fd43c205f admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:56.253214 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963972e-9e84-4c7f-86f8-efd68c23d3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:56.253916 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963972e-9e84-4c7f-86f8-efd68c23d3cd admin admin] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:56.254203 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963972e-9e84-4c7f-86f8-efd68c23d3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:56.254482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963972e-9e84-4c7f-86f8-efd68c23d3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:56.255839 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963972e-9e84-4c7f-86f8-efd68c23d3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:56.257470 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963972e-9e84-4c7f-86f8-efd68c23d3cd admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:56.269178 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1401/2871] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:04:56.273623 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9384d05-9259-4b4c-a07e-283b3b940ec2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:56.283591 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9384d05-9259-4b4c-a07e-283b3b940ec2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:56.288596 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8b5f6-6724-4dff-91d7-da02e32f8aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:56.289512 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8b5f6-6724-4dff-91d7-da02e32f8aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:56.289797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8b5f6-6724-4dff-91d7-da02e32f8aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:56.290071 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8b5f6-6724-4dff-91d7-da02e32f8aaf admin admin] PATH_INFO: `/v3/projects/5137218fb08e4f969b878d5cafd71eb9/users/ecfef5962b4242cda52c537e9440e3f9/roles/d94de411ec0a40be92c04950e6edcac7` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:56.307662 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8b5f6-6724-4dff-91d7-da02e32f8aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5137218fb08e4f969b878d5cafd71eb9, user_id=ecfef5962b4242cda52c537e9440e3f9, role_id=d94de411ec0a40be92c04950e6edcac7)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:56.310570 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8b5f6-6724-4dff-91d7-da02e32f8aaf admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:56.325058 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1471/2872] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:04:56 2022] PUT /identity/v3/projects/5137218fb08e4f969b878d5cafd71eb9/users/ecfef5962b4242cda52c537e9440e3f9/roles/d94de411ec0a40be92c04950e6edcac7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:04:56.331949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28510ba4-b41f-4703-80c5-0040ce37b980 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:56.332242 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28510ba4-b41f-4703-80c5-0040ce37b980 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:56.332537 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28510ba4-b41f-4703-80c5-0040ce37b980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:56.352400 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-28510ba4-b41f-4703-80c5-0040ce37b980 None None] MFA Rules not processed for user `629381bc156b476a94122bdd0a64b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:56.353636 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-28510ba4-b41f-4703-80c5-0040ce37b980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:56.418401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:56.430308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1402/2873] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:56.443199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8b5f6-6724-4dff-91d7-da02e32f8aaf admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:56.445033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-17a8b5f6-6724-4dff-91d7-da02e32f8aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:56.518749 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8b5f6-6724-4dff-91d7-da02e32f8aaf admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:56.522760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aff7b-3a27-4e83-b1d7-f333d30009a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:56.523420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aff7b-3a27-4e83-b1d7-f333d30009a3 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:56.523770 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aff7b-3a27-4e83-b1d7-f333d30009a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:56.523991 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aff7b-3a27-4e83-b1d7-f333d30009a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:56.533535 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aff7b-3a27-4e83-b1d7-f333d30009a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:56.534843 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aff7b-3a27-4e83-b1d7-f333d30009a3 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:56.545538 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1472/2874] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:05:01.554118 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c5e9a-aad4-4dc0-9b1b-711bdee6094d None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:01.554614 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c5e9a-aad4-4dc0-9b1b-711bdee6094d None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:01.554734 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c5e9a-aad4-4dc0-9b1b-711bdee6094d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:01.590026 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-e91c5e9a-aad4-4dc0-9b1b-711bdee6094d None None] MFA Rules not processed for user `ecfef5962b4242cda52c537e9440e3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:05:01.595861 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-e91c5e9a-aad4-4dc0-9b1b-711bdee6094d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:05:01.695428 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1403/2875] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:05:01.700013 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aff7b-3a27-4e83-b1d7-f333d30009a3 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:01.713244 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aff7b-3a27-4e83-b1d7-f333d30009a3 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:01.719422 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908b0aa-710c-4acf-862b-0d0742d0945a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:01.720280 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908b0aa-710c-4acf-862b-0d0742d0945a admin admin] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:01.720558 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908b0aa-710c-4acf-862b-0d0742d0945a admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:01.721019 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908b0aa-710c-4acf-862b-0d0742d0945a admin admin] PATH_INFO: `/v3/projects` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:01.721563 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908b0aa-710c-4acf-862b-0d0742d0945a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:01.723566 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908b0aa-710c-4acf-862b-0d0742d0945a admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:01.759626 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1473/2876] 10.222.0.70 () {60 vars in 1152 bytes} [Tue Jul 19 21:05:01 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:05:01.760888 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c5e9a-aad4-4dc0-9b1b-711bdee6094d None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:01.772949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c5e9a-aad4-4dc0-9b1b-711bdee6094d None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:01.856111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573e0f4-cb00-44f3-a3dd-00d6b70c5900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:01.856916 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573e0f4-cb00-44f3-a3dd-00d6b70c5900 admin admin] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:01.857252 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573e0f4-cb00-44f3-a3dd-00d6b70c5900 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:01.857651 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573e0f4-cb00-44f3-a3dd-00d6b70c5900 admin admin] PATH_INFO: `/v3/users` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:01.858901 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573e0f4-cb00-44f3-a3dd-00d6b70c5900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:01.860471 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573e0f4-cb00-44f3-a3dd-00d6b70c5900 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:01.889002 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1404/2877] 10.222.0.70 () {60 vars in 1143 bytes} [Tue Jul 19 21:05:01 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:05:01.891222 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908b0aa-710c-4acf-862b-0d0742d0945a admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:01.906772 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908b0aa-710c-4acf-862b-0d0742d0945a admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:01.913215 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f301d1-822c-45cc-8589-3efadd02cdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:01.913945 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f301d1-822c-45cc-8589-3efadd02cdc2 admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:01.914236 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f301d1-822c-45cc-8589-3efadd02cdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:01.914582 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f301d1-822c-45cc-8589-3efadd02cdc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:01.915278 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f301d1-822c-45cc-8589-3efadd02cdc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:01.917077 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f301d1-822c-45cc-8589-3efadd02cdc2 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:01.930256 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1474/2878] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:05:01.934543 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573e0f4-cb00-44f3-a3dd-00d6b70c5900 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:01.946602 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573e0f4-cb00-44f3-a3dd-00d6b70c5900 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:01.951748 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653261b-188c-4e87-9e0c-85ca7feeee40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:01.952735 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653261b-188c-4e87-9e0c-85ca7feeee40 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:01.953034 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653261b-188c-4e87-9e0c-85ca7feeee40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:01.953321 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653261b-188c-4e87-9e0c-85ca7feeee40 admin admin] PATH_INFO: `/v3/projects/d0b3f27ba7b54982afb17a62003c0aa0/users/ecffcdd879204f3ba701b6fc619582e7/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:01.974733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653261b-188c-4e87-9e0c-85ca7feeee40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b3f27ba7b54982afb17a62003c0aa0, user_id=ecffcdd879204f3ba701b6fc619582e7, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:01.977616 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653261b-188c-4e87-9e0c-85ca7feeee40 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:01.999711 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1405/2879] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:05:01 2022] PUT /identity/v3/projects/d0b3f27ba7b54982afb17a62003c0aa0/users/ecffcdd879204f3ba701b6fc619582e7/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:05:02.003103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f301d1-822c-45cc-8589-3efadd02cdc2 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:02.012646 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:05:02.028009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f301d1-822c-45cc-8589-3efadd02cdc2 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:02.122214 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7049bc-2e3f-460d-bb9f-99d7e58c2afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:02.122892 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7049bc-2e3f-460d-bb9f-99d7e58c2afc admin admin] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:02.123173 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7049bc-2e3f-460d-bb9f-99d7e58c2afc admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:02.123448 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7049bc-2e3f-460d-bb9f-99d7e58c2afc admin admin] PATH_INFO: `/v3/roles` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:02.124174 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7049bc-2e3f-460d-bb9f-99d7e58c2afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:02.125824 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7049bc-2e3f-460d-bb9f-99d7e58c2afc admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:02.136872 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1475/2880] 10.222.0.70 () {58 vars in 1121 bytes} [Tue Jul 19 21:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:05:02.141331 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653261b-188c-4e87-9e0c-85ca7feeee40 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:02.154497 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653261b-188c-4e87-9e0c-85ca7feeee40 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:02.159436 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5571a-7956-4bbd-9234-e74f2a8bec39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:02.160335 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5571a-7956-4bbd-9234-e74f2a8bec39 admin admin] REQUEST_METHOD: `PUT` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:02.160644 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5571a-7956-4bbd-9234-e74f2a8bec39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:02.160919 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5571a-7956-4bbd-9234-e74f2a8bec39 admin admin] PATH_INFO: `/v3/projects/d0b3f27ba7b54982afb17a62003c0aa0/users/ecffcdd879204f3ba701b6fc619582e7/roles/85420c68300642aabfd730fb8fef8347` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:02.163048 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5571a-7956-4bbd-9234-e74f2a8bec39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b3f27ba7b54982afb17a62003c0aa0, user_id=ecffcdd879204f3ba701b6fc619582e7, role_id=85420c68300642aabfd730fb8fef8347)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:02.165715 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5571a-7956-4bbd-9234-e74f2a8bec39 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:02.177327 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1406/2881] 10.222.0.70 () {60 vars in 1482 bytes} [Tue Jul 19 21:05:02 2022] PUT /identity/v3/projects/d0b3f27ba7b54982afb17a62003c0aa0/users/ecffcdd879204f3ba701b6fc619582e7/roles/85420c68300642aabfd730fb8fef8347 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:05:07.474958 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854ffdc-693c-4abb-b1b5-38fb5c654123 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:07.475993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854ffdc-693c-4abb-b1b5-38fb5c654123 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:07.475993 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854ffdc-693c-4abb-b1b5-38fb5c654123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:07.511397 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-6854ffdc-693c-4abb-b1b5-38fb5c654123 None None] MFA Rules not processed for user `ecffcdd879204f3ba701b6fc619582e7`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:05:07.512676 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-6854ffdc-693c-4abb-b1b5-38fb5c654123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:05:07.607317 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1476/2882] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:05:07.622797 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5571a-7956-4bbd-9234-e74f2a8bec39 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:07.624846 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-0eb5571a-7956-4bbd-9234-e74f2a8bec39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:05:07.676609 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:05:07.710111 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5571a-7956-4bbd-9234-e74f2a8bec39 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:07.715886 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52e6bf-4811-4901-ba42-6c6611557bf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:07.716554 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52e6bf-4811-4901-ba42-6c6611557bf0 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:07.716842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52e6bf-4811-4901-ba42-6c6611557bf0 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:07.717122 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52e6bf-4811-4901-ba42-6c6611557bf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:07.785396 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52e6bf-4811-4901-ba42-6c6611557bf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:07.787289 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52e6bf-4811-4901-ba42-6c6611557bf0 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:07.799024 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1407/2883] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:05:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:05:09.002005 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a4229-66e5-4c33-9178-8944b042d357 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:09.002577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a4229-66e5-4c33-9178-8944b042d357 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:09.002577 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a4229-66e5-4c33-9178-8944b042d357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:09.023894 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-7f7a4229-66e5-4c33-9178-8944b042d357 None None] MFA Rules not processed for user `ecfef5962b4242cda52c537e9440e3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:05:09.025168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-7f7a4229-66e5-4c33-9178-8944b042d357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:05:09.040564 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1477/2884] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:05:09.469477 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52e6bf-4811-4901-ba42-6c6611557bf0 service nova] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:09.471529 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ae52e6bf-4811-4901-ba42-6c6611557bf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:05:09.588164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52e6bf-4811-4901-ba42-6c6611557bf0 service nova] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:09.593723 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d7836-1de8-49bd-9a39-f66af9f779af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:09.594697 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7836-1de8-49bd-9a39-f66af9f779af service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:09.595103 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7836-1de8-49bd-9a39-f66af9f779af service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:09.595219 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7836-1de8-49bd-9a39-f66af9f779af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:09.597582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-242d7836-1de8-49bd-9a39-f66af9f779af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:05:09.609375 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d7836-1de8-49bd-9a39-f66af9f779af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:09.610907 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d7836-1de8-49bd-9a39-f66af9f779af service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:09.622991 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1408/2885] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:05:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:05:19.018962 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=65 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:05:19.617011 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=64 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:05:50.697745 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a4229-66e5-4c33-9178-8944b042d357 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:50.708149 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a4229-66e5-4c33-9178-8944b042d357 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:50.711822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b304f-85b9-4377-b677-cd1628daae48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:50.712463 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b304f-85b9-4377-b677-cd1628daae48 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:50.712744 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b304f-85b9-4377-b677-cd1628daae48 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:50.713020 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b304f-85b9-4377-b677-cd1628daae48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:50.722328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b304f-85b9-4377-b677-cd1628daae48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:50.723747 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b304f-85b9-4377-b677-cd1628daae48 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:50.733147 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1478/2886] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:05:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:05:52.392119 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d7836-1de8-49bd-9a39-f66af9f779af service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:52.405771 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d7836-1de8-49bd-9a39-f66af9f779af service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:52.411860 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55031549-b0ce-493b-bf00-ffe5150aa2a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:52.412501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55031549-b0ce-493b-bf00-ffe5150aa2a4 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:52.412784 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55031549-b0ce-493b-bf00-ffe5150aa2a4 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:52.413059 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55031549-b0ce-493b-bf00-ffe5150aa2a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:52.414842 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-55031549-b0ce-493b-bf00-ffe5150aa2a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:05:52.428515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55031549-b0ce-493b-bf00-ffe5150aa2a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:52.430137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55031549-b0ce-493b-bf00-ffe5150aa2a4 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:52.442370 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1409/2887] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:05:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:00.728933 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=6 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:02.436932 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=6 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:16.728199 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b304f-85b9-4377-b677-cd1628daae48 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:16.729614 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-b66b304f-85b9-4377-b677-cd1628daae48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:16.806398 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b304f-85b9-4377-b677-cd1628daae48 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:16.810092 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f978e85-fb1d-4ff0-941e-9ac84cd465aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a75951e9512c420992a4fd6c1a6b1176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:16.810864 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978e85-fb1d-4ff0-941e-9ac84cd465aa service cinder] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:16.811026 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978e85-fb1d-4ff0-941e-9ac84cd465aa service cinder] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:16.811307 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978e85-fb1d-4ff0-941e-9ac84cd465aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:16.813299 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4f978e85-fb1d-4ff0-941e-9ac84cd465aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:16.888311 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f978e85-fb1d-4ff0-941e-9ac84cd465aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:16.889800 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f978e85-fb1d-4ff0-941e-9ac84cd465aa service cinder] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:16.898875 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1479/2888] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:07:26.894616 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=63 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:29.000349 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c13c7-e43a-43ce-bd1a-b969996d41cc None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:29.000900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c13c7-e43a-43ce-bd1a-b969996d41cc None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:29.000900 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c13c7-e43a-43ce-bd1a-b969996d41cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:29.028224 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-397c13c7-e43a-43ce-bd1a-b969996d41cc None None] MFA Rules not processed for user `cf1d05ee73d4414eb51e2a7be838e28f`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:29.029459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-397c13c7-e43a-43ce-bd1a-b969996d41cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:29.036825 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1410/2889] 10.222.0.70 () {58 vars in 957 bytes} [Tue Jul 19 21:07:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:07:30.810033 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f978e85-fb1d-4ff0-941e-9ac84cd465aa service cinder] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:30.811586 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-4f978e85-fb1d-4ff0-941e-9ac84cd465aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:30.912881 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f978e85-fb1d-4ff0-941e-9ac84cd465aa service cinder] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:30.918469 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7c41-ad38-49c0-be84-ee410a640e3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:30.919137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7c41-ad38-49c0-be84-ee410a640e3e service placement] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:30.919431 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7c41-ad38-49c0-be84-ee410a640e3e service placement] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:30.919694 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7c41-ad38-49c0-be84-ee410a640e3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:30.923257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-0ada7c41-ad38-49c0-be84-ee410a640e3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:30.935952 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada7c41-ad38-49c0-be84-ee410a640e3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:30.937608 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada7c41-ad38-49c0-be84-ee410a640e3e service placement] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:30.949073 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1480/2890] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:07:36.524835 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c13c7-e43a-43ce-bd1a-b969996d41cc None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:36.529215 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-397c13c7-e43a-43ce-bd1a-b969996d41cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:36.566302 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c13c7-e43a-43ce-bd1a-b969996d41cc None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:36.571926 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afa43c-ae03-4884-b6d1-93d19a64787b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:36.572588 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afa43c-ae03-4884-b6d1-93d19a64787b service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:36.572872 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afa43c-ae03-4884-b6d1-93d19a64787b service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:36.573159 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afa43c-ae03-4884-b6d1-93d19a64787b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:36.575088 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-f3afa43c-ae03-4884-b6d1-93d19a64787b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:36.589452 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afa43c-ae03-4884-b6d1-93d19a64787b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:36.591295 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afa43c-ae03-4884-b6d1-93d19a64787b service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:36.605098 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1411/2891] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:07:40.943976 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=39 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:46.599480 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=16 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:47.932555 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7c41-ad38-49c0-be84-ee410a640e3e service placement] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:47.948054 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7c41-ad38-49c0-be84-ee410a640e3e service placement] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:47.955925 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8548d8-b6ba-4eca-9080-c99d04634b00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:47.956597 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8548d8-b6ba-4eca-9080-c99d04634b00 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:47.956884 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8548d8-b6ba-4eca-9080-c99d04634b00 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:47.957190 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8548d8-b6ba-4eca-9080-c99d04634b00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:47.959283 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-1a8548d8-b6ba-4eca-9080-c99d04634b00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:47.974206 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8548d8-b6ba-4eca-9080-c99d04634b00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:47.975804 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8548d8-b6ba-4eca-9080-c99d04634b00 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:47.990406 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1481/2892] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:07:50.810263 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afa43c-ae03-4884-b6d1-93d19a64787b service nova] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:50.820198 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afa43c-ae03-4884-b6d1-93d19a64787b service nova] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:50.823875 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ad53e-d200-4d3d-9d31-11918efe7167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:50.824515 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad53e-d200-4d3d-9d31-11918efe7167 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:50.824799 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad53e-d200-4d3d-9d31-11918efe7167 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:50.825084 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad53e-d200-4d3d-9d31-11918efe7167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:50.826889 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-618ad53e-d200-4d3d-9d31-11918efe7167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:50.890482 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ad53e-d200-4d3d-9d31-11918efe7167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:50.892387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ad53e-d200-4d3d-9d31-11918efe7167 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:50.902031 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1412/2893] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:07:57.982401 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=6 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:08:00.897717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=30 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:08:12.692981 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8548d8-b6ba-4eca-9080-c99d04634b00 service nova] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:08:12.715184 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8548d8-b6ba-4eca-9080-c99d04634b00 service nova] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:08:12.721048 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477de724-fe25-44d2-b737-56b6042a93a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:08:12.721792 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477de724-fe25-44d2-b737-56b6042a93a9 service placement] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:08:12.722080 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477de724-fe25-44d2-b737-56b6042a93a9 service placement] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:08:12.722367 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477de724-fe25-44d2-b737-56b6042a93a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:08:12.724218 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-477de724-fe25-44d2-b737-56b6042a93a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:08:12.750286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477de724-fe25-44d2-b737-56b6042a93a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:08:12.751861 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477de724-fe25-44d2-b737-56b6042a93a9 service placement] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:08:12.763349 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1482/2894] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:08:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:08:22.758306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=15 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:08:23.987540 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ad53e-d200-4d3d-9d31-11918efe7167 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:08:23.996974 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ad53e-d200-4d3d-9d31-11918efe7167 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:08:24.000668 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15961b22-1c3e-4267-9623-efe8c9e63d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:08:24.001862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15961b22-1c3e-4267-9623-efe8c9e63d58 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:08:24.001862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15961b22-1c3e-4267-9623-efe8c9e63d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:08:24.002076 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15961b22-1c3e-4267-9623-efe8c9e63d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:08:24.003733 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-15961b22-1c3e-4267-9623-efe8c9e63d58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:08:24.014878 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15961b22-1c3e-4267-9623-efe8c9e63d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:08:24.017408 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15961b22-1c3e-4267-9623-efe8c9e63d58 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:08:24.029507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1413/2895] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:08:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:08:24.141491 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0c5e7-9459-411c-bb4e-4d3566010e4d None None] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:08:24.142122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0c5e7-9459-411c-bb4e-4d3566010e4d None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:08:24.142122 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0c5e7-9459-411c-bb4e-4d3566010e4d None None] PATH_INFO: `/` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:08:24.143666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1483/2896] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 21:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 21:08:24.149237 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a014a83-8d97-4cfc-a9fd-4720c92042b7 None None] REQUEST_METHOD: `POST` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:08:24.149571 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a014a83-8d97-4cfc-a9fd-4720c92042b7 None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:08:24.149854 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a014a83-8d97-4cfc-a9fd-4720c92042b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:08:24.201013 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.auth.core [None req-8a014a83-8d97-4cfc-a9fd-4720c92042b7 None None] MFA Rules not processed for user `fb186ee2e3c94ca7ad203c9c0e32e498`. Rule list: `[]` (Enabled: `True`). {{(pid=92311) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:08:24.202356 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8a014a83-8d97-4cfc-a9fd-4720c92042b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:08:24.209512 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1414/2897] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 21:08:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:08:24.221867 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0c5e7-9459-411c-bb4e-4d3566010e4d None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:08:24.233231 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0c5e7-9459-411c-bb4e-4d3566010e4d None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:08:24.237142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e249f61-a3d3-4d71-80af-b568b746a33d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:08:24.237828 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e249f61-a3d3-4d71-80af-b568b746a33d service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:08:24.238117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e249f61-a3d3-4d71-80af-b568b746a33d service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:08:24.238399 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e249f61-a3d3-4d71-80af-b568b746a33d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:08:24.246841 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e249f61-a3d3-4d71-80af-b568b746a33d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:08:24.248265 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e249f61-a3d3-4d71-80af-b568b746a33d service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:08:24.257142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1484/2898] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:08:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:08:24.342427 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91a6e7-6399-4b7f-b6ee-ce46e401590e None None] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:08:24.342717 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91a6e7-6399-4b7f-b6ee-ce46e401590e None None] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:08:24.342995 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91a6e7-6399-4b7f-b6ee-ce46e401590e None None] PATH_INFO: `/` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:08:24.344700 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1415/2899] 10.222.0.70 () {54 vars in 921 bytes} [Tue Jul 19 21:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 21:08:24.350010 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743ae72-038e-4134-9e25-e97e6e8c6c87 None None] REQUEST_METHOD: `POST` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:08:24.350319 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743ae72-038e-4134-9e25-e97e6e8c6c87 None None] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:08:24.350631 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743ae72-038e-4134-9e25-e97e6e8c6c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:08:24.390303 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.auth.core [None req-8743ae72-038e-4134-9e25-e97e6e8c6c87 None None] MFA Rules not processed for user `e1d09e168fc54f1da06b719989ae8ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=92310) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:08:24.391510 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-8743ae72-038e-4134-9e25-e97e6e8c6c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:08:24.400629 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1485/2900] 10.222.0.70 () {58 vars in 1018 bytes} [Tue Jul 19 21:08:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:08:24.411550 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91a6e7-6399-4b7f-b6ee-ce46e401590e None None] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:08:24.422804 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91a6e7-6399-4b7f-b6ee-ce46e401590e None None] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:08:24.426994 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8903d-141b-4a75-bfd1-b4f2dbfa076b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:08:24.427623 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8903d-141b-4a75-bfd1-b4f2dbfa076b service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:08:24.427912 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8903d-141b-4a75-bfd1-b4f2dbfa076b service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:08:24.428190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8903d-141b-4a75-bfd1-b4f2dbfa076b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:08:24.438275 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8903d-141b-4a75-bfd1-b4f2dbfa076b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:08:24.439731 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8903d-141b-4a75-bfd1-b4f2dbfa076b service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:08:24.450369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1416/2901] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:08:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:08:34.387813 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=15 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:08:34.445725 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=25 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:09:56.680393 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743ae72-038e-4134-9e25-e97e6e8c6c87 None None] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:09:56.701822 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743ae72-038e-4134-9e25-e97e6e8c6c87 None None] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:09:56.708613 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be1120-583e-4c2b-b59e-f3dd3e16893d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:09:56.709333 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be1120-583e-4c2b-b59e-f3dd3e16893d service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:56.709609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be1120-583e-4c2b-b59e-f3dd3e16893d service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:56.709885 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be1120-583e-4c2b-b59e-f3dd3e16893d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:56.711949 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-52be1120-583e-4c2b-b59e-f3dd3e16893d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:56.740481 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be1120-583e-4c2b-b59e-f3dd3e16893d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:09:56.742012 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be1120-583e-4c2b-b59e-f3dd3e16893d service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:09:56.765245 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1486/2902] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:09:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:09:57.949882 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8903d-141b-4a75-bfd1-b4f2dbfa076b service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:09:57.973662 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8903d-141b-4a75-bfd1-b4f2dbfa076b service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:09:57.977506 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc6ef6-79d4-4afb-8f37-1e40a7e8d825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:09:57.978152 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc6ef6-79d4-4afb-8f37-1e40a7e8d825 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:57.978431 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc6ef6-79d4-4afb-8f37-1e40a7e8d825 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:57.978703 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc6ef6-79d4-4afb-8f37-1e40a7e8d825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:57.980688 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-73bc6ef6-79d4-4afb-8f37-1e40a7e8d825 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:57.994350 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc6ef6-79d4-4afb-8f37-1e40a7e8d825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:09:57.995757 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc6ef6-79d4-4afb-8f37-1e40a7e8d825 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:09:58.010369 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1417/2903] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:09:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:10:06.762482 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=19 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:10:08.002137 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=10 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:10:09.298141 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be1120-583e-4c2b-b59e-f3dd3e16893d service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:10:09.308009 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be1120-583e-4c2b-b59e-f3dd3e16893d service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:10:09.311683 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aa0f0-36bf-488d-b31d-b88d3396e8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:10:09.312328 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aa0f0-36bf-488d-b31d-b88d3396e8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:10:09.312620 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aa0f0-36bf-488d-b31d-b88d3396e8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:10:09.312902 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aa0f0-36bf-488d-b31d-b88d3396e8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:10:09.314712 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-322aa0f0-36bf-488d-b31d-b88d3396e8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:10:09.401691 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322aa0f0-36bf-488d-b31d-b88d3396e8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:10:09.403117 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322aa0f0-36bf-488d-b31d-b88d3396e8e9 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:10:09.422583 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1487/2904] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:10:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:10:12.297679 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc6ef6-79d4-4afb-8f37-1e40a7e8d825 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:10:12.299415 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-73bc6ef6-79d4-4afb-8f37-1e40a7e8d825 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:10:12.320157 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc6ef6-79d4-4afb-8f37-1e40a7e8d825 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:10:12.325430 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daaddc-194d-4e6e-944a-a86673ad5da9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:10:12.326081 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daaddc-194d-4e6e-944a-a86673ad5da9 service placement] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:10:12.326391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daaddc-194d-4e6e-944a-a86673ad5da9 service placement] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:10:12.326815 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59daaddc-194d-4e6e-944a-a86673ad5da9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:10:12.328583 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-59daaddc-194d-4e6e-944a-a86673ad5da9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:10:12.341047 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daaddc-194d-4e6e-944a-a86673ad5da9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:10:12.343447 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59daaddc-194d-4e6e-944a-a86673ad5da9 service placement] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:10:12.357881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1418/2905] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:10:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:10:15.729087 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aa0f0-36bf-488d-b31d-b88d3396e8e9 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:10:15.743155 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aa0f0-36bf-488d-b31d-b88d3396e8e9 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:10:15.751427 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b0f5a-2721-48c3-9f4e-d2b0e6b2e7ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:10:15.752084 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b0f5a-2721-48c3-9f4e-d2b0e6b2e7ab service placement] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:10:15.752368 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b0f5a-2721-48c3-9f4e-d2b0e6b2e7ab service placement] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:10:15.752644 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b0f5a-2721-48c3-9f4e-d2b0e6b2e7ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:10:15.754466 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-bf7b0f5a-2721-48c3-9f4e-d2b0e6b2e7ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:10:15.767089 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b0f5a-2721-48c3-9f4e-d2b0e6b2e7ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:10:15.769123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b0f5a-2721-48c3-9f4e-d2b0e6b2e7ab service placement] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:10:15.780457 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1488/2906] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:10:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:10:15.795834 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daaddc-194d-4e6e-944a-a86673ad5da9 service placement] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:10:15.797710 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-59daaddc-194d-4e6e-944a-a86673ad5da9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:10:15.814979 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59daaddc-194d-4e6e-944a-a86673ad5da9 service placement] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:10:15.819589 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:10:15.820472 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:10:15.820756 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:10:15.821036 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] PATH_INFO: `/v3/users/cf1d05ee73d4414eb51e2a7be838e28f` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:10:15.822299 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1d05ee73d4414eb51e2a7be838e28f)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:10:15.824292 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:10:15.873646 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1d05ee73d4414eb51e2a7be838e28f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:15.881450 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1d05ee73d4414eb51e2a7be838e28f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:15.888475 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1d05ee73d4414eb51e2a7be838e28f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:15.900246 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1d05ee73d4414eb51e2a7be838e28f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:15.900821 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1d05ee73d4414eb51e2a7be838e28f'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:15.906961 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1419/2907] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:10:15 2022] DELETE /identity/v3/users/cf1d05ee73d4414eb51e2a7be838e28f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:10:16.089666 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b0f5a-2721-48c3-9f4e-d2b0e6b2e7ab service placement] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:10:16.104982 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b0f5a-2721-48c3-9f4e-d2b0e6b2e7ab service placement] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:10:16.183576 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:10:16.184452 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:10:16.184737 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:10:16.185123 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] PATH_INFO: `/v3/projects/60fdf04148f548f1b9fe86ed038e839f` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:10:16.186255 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60fdf04148f548f1b9fe86ed038e839f)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:10:16.188168 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:10:16.237144 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60fdf04148f548f1b9fe86ed038e839f', 'request_id': 'req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.240776 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:10:16.245142 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60fdf04148f548f1b9fe86ed038e839f', 'request_id': 'req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.252292 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60fdf04148f548f1b9fe86ed038e839f', 'request_id': 'req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.263735 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] The token cache is being invalidate because project 60fdf04148f548f1b9fe86ed038e839f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:10:16.264024 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.264671 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.266320 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1489/2908] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:10:16 2022] DELETE /identity/v3/projects/60fdf04148f548f1b9fe86ed038e839f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:10:16.270984 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:10:16.272387 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:10:16.285558 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86fc92-4758-4c8d-a392-4cad9e3b1159 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:10:16.343884 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:10:16.345018 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:10:16.345308 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:10:16.345649 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] PATH_INFO: `/v3/users/1908f29da22e47fc9071d1cfb495c561` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:10:16.346862 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1908f29da22e47fc9071d1cfb495c561)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:10:16.349469 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:10:16.397857 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1908f29da22e47fc9071d1cfb495c561'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.404885 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1908f29da22e47fc9071d1cfb495c561'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.410233 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:10:16.412582 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1908f29da22e47fc9071d1cfb495c561'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.426379 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1908f29da22e47fc9071d1cfb495c561'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.427164 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1908f29da22e47fc9071d1cfb495c561'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.433265 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1420/2909] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:10:16 2022] DELETE /identity/v3/users/1908f29da22e47fc9071d1cfb495c561 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:10:16.609146 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:10:16.610947 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:10:16.633124 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8832a-5bf4-49e4-bc3e-b7e1b4e90864 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:10:16.690609 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:10:16.691423 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:10:16.691706 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:10:16.691981 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] PATH_INFO: `/v3/projects/0b9d6363aed94f1e8407fa51bf167f08` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:10:16.693699 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9d6363aed94f1e8407fa51bf167f08)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:10:16.695598 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:10:16.744831 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9d6363aed94f1e8407fa51bf167f08', 'request_id': 'req-a0092877-2cba-4bf4-9b4b-a386b4b18399'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.752137 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9d6363aed94f1e8407fa51bf167f08', 'request_id': 'req-a0092877-2cba-4bf4-9b4b-a386b4b18399'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.758653 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9d6363aed94f1e8407fa51bf167f08', 'request_id': 'req-a0092877-2cba-4bf4-9b4b-a386b4b18399'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.769935 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] The token cache is being invalidate because project 0b9d6363aed94f1e8407fa51bf167f08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:10:16.770227 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.771544 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:10:16.773088 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1490/2910] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:10:16 2022] DELETE /identity/v3/projects/0b9d6363aed94f1e8407fa51bf167f08 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:10:26.429729 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:10:26.766930 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:11:17.373949 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:11:17.376075 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:11:17.447476 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea82b9-4d44-4d6b-8dfd-a86cc7ea92ae admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:11:17.496553 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83f059-a151-49d6-bd6d-be6f81779af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:11:17.497223 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83f059-a151-49d6-bd6d-be6f81779af3 service neutron] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:11:17.497547 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83f059-a151-49d6-bd6d-be6f81779af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:11:17.497831 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83f059-a151-49d6-bd6d-be6f81779af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:11:17.499772 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ae83f059-a151-49d6-bd6d-be6f81779af3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:11:17.510374 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83f059-a151-49d6-bd6d-be6f81779af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:11:17.511793 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83f059-a151-49d6-bd6d-be6f81779af3 service neutron] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:11:17.519852 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1421/2911] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:11:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:11:27.516218 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=51 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:12:06.519326 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:12:06.520655 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:12:06.593286 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0092877-2cba-4bf4-9b4b-a386b4b18399 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:12:06.599097 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc9c24-5a23-4052-ab13-5396f174a489 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:12:06.599742 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc9c24-5a23-4052-ab13-5396f174a489 service nova] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:12:06.600027 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc9c24-5a23-4052-ab13-5396f174a489 service nova] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:12:06.600306 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc9c24-5a23-4052-ab13-5396f174a489 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:12:06.602081 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-01fc9c24-5a23-4052-ab13-5396f174a489 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:12:06.661063 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc9c24-5a23-4052-ab13-5396f174a489 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:12:06.662766 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc9c24-5a23-4052-ab13-5396f174a489 service nova] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:12:06.671722 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1491/2912] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:12:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:12:07.640391 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83f059-a151-49d6-bd6d-be6f81779af3 service neutron] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:12:07.656074 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83f059-a151-49d6-bd6d-be6f81779af3 service neutron] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:12:07.661998 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171ae6d-5a0a-4887-9cfe-8e8b749b5961 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d09e168fc54f1da06b719989ae8ef0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:12:07.662730 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171ae6d-5a0a-4887-9cfe-8e8b749b5961 service nova] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:12:07.663095 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171ae6d-5a0a-4887-9cfe-8e8b749b5961 service nova] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:12:07.663501 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171ae6d-5a0a-4887-9cfe-8e8b749b5961 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:12:07.665459 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8171ae6d-5a0a-4887-9cfe-8e8b749b5961 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:12:07.679421 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171ae6d-5a0a-4887-9cfe-8e8b749b5961 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:12:07.681109 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171ae6d-5a0a-4887-9cfe-8e8b749b5961 service nova] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:12:07.693163 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1422/2913] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:12:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:12:11.525372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc9c24-5a23-4052-ab13-5396f174a489 service nova] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:12:11.525372 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-01fc9c24-5a23-4052-ab13-5396f174a489 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:12:11.535287 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc9c24-5a23-4052-ab13-5396f174a489 service nova] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:12:11.539447 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14645285-96fc-4e2f-9756-d28eec792964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb186ee2e3c94ca7ad203c9c0e32e498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:12:11.540094 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14645285-96fc-4e2f-9756-d28eec792964 service neutron] REQUEST_METHOD: `GET` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:12:11.540384 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14645285-96fc-4e2f-9756-d28eec792964 service neutron] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:12:11.540665 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14645285-96fc-4e2f-9756-d28eec792964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:12:11.542460 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-14645285-96fc-4e2f-9756-d28eec792964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:12:11.615798 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14645285-96fc-4e2f-9756-d28eec792964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:12:11.617257 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14645285-96fc-4e2f-9756-d28eec792964 service neutron] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:12:11.627166 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1492/2914] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:12:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:12:17.688760 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=6 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:12:17.785532 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171ae6d-5a0a-4887-9cfe-8e8b749b5961 service nova] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:12:17.787090 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-8171ae6d-5a0a-4887-9cfe-8e8b749b5961 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:12:17.800704 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171ae6d-5a0a-4887-9cfe-8e8b749b5961 service nova] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:12:17.806134 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:12:17.807017 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:12:17.807307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:12:17.807586 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] PATH_INFO: `/v3/users/ecfef5962b4242cda52c537e9440e3f9` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:12:17.808785 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecfef5962b4242cda52c537e9440e3f9)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:12:17.810840 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:12:17.862474 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecfef5962b4242cda52c537e9440e3f9'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:17.871827 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecfef5962b4242cda52c537e9440e3f9'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:17.879921 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecfef5962b4242cda52c537e9440e3f9'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:17.893647 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecfef5962b4242cda52c537e9440e3f9'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:17.894276 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecfef5962b4242cda52c537e9440e3f9'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:17.901401 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1423/2915] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:12:17 2022] DELETE /identity/v3/users/ecfef5962b4242cda52c537e9440e3f9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:12:18.089103 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14645285-96fc-4e2f-9756-d28eec792964 service neutron] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:12:18.101484 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14645285-96fc-4e2f-9756-d28eec792964 service neutron] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:12:18.138285 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=100 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:12:18.168420 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:12:18.169211 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:12:18.169520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:12:18.169797 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] PATH_INFO: `/v3/projects/5137218fb08e4f969b878d5cafd71eb9` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:12:18.170978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5137218fb08e4f969b878d5cafd71eb9)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:12:18.172886 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:12:18.221715 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5137218fb08e4f969b878d5cafd71eb9', 'request_id': 'req-5077c78d-0c2b-4ab5-b808-072e3f516b60'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.230995 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5137218fb08e4f969b878d5cafd71eb9', 'request_id': 'req-5077c78d-0c2b-4ab5-b808-072e3f516b60'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.236856 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5137218fb08e4f969b878d5cafd71eb9', 'request_id': 'req-5077c78d-0c2b-4ab5-b808-072e3f516b60'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.248423 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] The token cache is being invalidate because project 5137218fb08e4f969b878d5cafd71eb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:12:18.249059 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.249968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.251830 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1493/2916] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:12:18 2022] DELETE /identity/v3/projects/5137218fb08e4f969b878d5cafd71eb9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:12:18.256384 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:12:18.257886 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:12:18.271190 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce436301-fd65-40cf-80a1-384f3e0e614b admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:12:18.327916 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:12:18.328791 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:12:18.329069 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:12:18.329338 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] PATH_INFO: `/v3/users/ecffcdd879204f3ba701b6fc619582e7` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:12:18.330559 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecffcdd879204f3ba701b6fc619582e7)` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:12:18.332449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:12:18.379341 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecffcdd879204f3ba701b6fc619582e7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.388510 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecffcdd879204f3ba701b6fc619582e7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.395836 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecffcdd879204f3ba701b6fc619582e7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.403691 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:12:18.408727 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecffcdd879204f3ba701b6fc619582e7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.409449 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.notifications [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecffcdd879204f3ba701b6fc619582e7'} {{(pid=92311) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.417311 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1424/2917] 10.222.0.70 () {58 vars in 1223 bytes} [Tue Jul 19 21:12:18 2022] DELETE /identity/v3/users/ecffcdd879204f3ba701b6fc619582e7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:12:18.598409 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] Authenticating user token {{(pid=92310) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:12:18.600520 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.fernet_utils [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92310) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:12:18.615978 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077c78d-0c2b-4ab5-b808-072e3f516b60 admin admin] Validating token access rules against request {{(pid=92310) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:12:18.675238 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '629381bc156b476a94122bdd0a64b60d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9ddf9c6d32c459d8f135c30485e0f10', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92310) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:12:18.675968 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:12:18.676254 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:12:18.676531 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] PATH_INFO: `/v3/projects/d0b3f27ba7b54982afb17a62003c0aa0` {{(pid=92310) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:12:18.677794 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0b3f27ba7b54982afb17a62003c0aa0)` {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:12:18.679760 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] RBAC: Authorization granted {{(pid=92310) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:12:18.726934 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0b3f27ba7b54982afb17a62003c0aa0', 'request_id': 'req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.734382 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0b3f27ba7b54982afb17a62003c0aa0', 'request_id': 'req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.739002 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:12:18.740216 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0b3f27ba7b54982afb17a62003c0aa0', 'request_id': 'req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12'} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.753075 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] The token cache is being invalidate because project d0b3f27ba7b54982afb17a62003c0aa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92310) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:12:18.753381 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.754029 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG keystone.notifications [None req-625d054e-aa8d-4f01-a394-b1b6aa9b2b12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92310) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:12:18.756143 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: [pid: 92310|app: 0|req: 1494/2918] 10.222.0.70 () {58 vars in 1232 bytes} [Tue Jul 19 21:12:18 2022] DELETE /identity/v3/projects/d0b3f27ba7b54982afb17a62003c0aa0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:12:28.416511 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:12:28.750258 dv-cinder-799871-37-iscsi devstack@keystone.service[92310]: DEBUG dbcounter [-] [92310] Writing DB stats keystone:SELECT=4 {{(pid=92310) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:14.608167 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] Authenticating user token {{(pid=92311) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:14.609643 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:14.676881 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa7b6c-45fa-44cb-9fec-d6129879d573 admin admin] Validating token access rules against request {{(pid=92311) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:14.736507 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b0197-3ad3-4e77-9b90-6a22adaabe54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f27871b39b9b4edaa512bd5b7467174a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7709322f45334773aaccd8472915ec38', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92311) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:14.737083 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b0197-3ad3-4e77-9b90-6a22adaabe54 service placement] REQUEST_METHOD: `GET` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:14.737390 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b0197-3ad3-4e77-9b90-6a22adaabe54 service placement] SCRIPT_NAME: `/identity` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:14.737681 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b0197-3ad3-4e77-9b90-6a22adaabe54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92311) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:14.739466 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.fernet_utils [None req-c46b0197-3ad3-4e77-9b90-6a22adaabe54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92311) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:14.750699 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b0197-3ad3-4e77-9b90-6a22adaabe54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:14.752307 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b0197-3ad3-4e77-9b90-6a22adaabe54 service placement] RBAC: Authorization granted {{(pid=92311) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:14.762673 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: [pid: 92311|app: 0|req: 1425/2919] 10.222.0.70 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:13:24.758453 dv-cinder-799871-37-iscsi devstack@keystone.service[92311]: DEBUG dbcounter [-] [92311] Writing DB stats keystone:SELECT=58 {{(pid=92311) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}